[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786240941968412-1795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:02:22 dist-test-slave-ss4n krb5kdc[1804](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:02:22 dist-test-slave-ss4n krb5kdc[1804](info): set up 2 sockets
Aug 09 02:02:22 dist-test-slave-ss4n krb5kdc[1804](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260809 02:02:23.192613 1795 mini_kdc.cc:121] Time spent starting KDC: real 1.150s user 0.004s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786240941968412-1795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786240941968412-1795-0/krb5kdc/test-user.keytab.
Aug 09 02:02:23 dist-test-slave-ss4n krb5kdc[1804](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240943, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:02:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:02:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.192.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.192.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.192.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab.
Entry for principal kudu/127.1.192.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab.
I20260809 02:02:23.341614 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786240941968412-1795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786240941968412-1795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786240941968412-1795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.192.254:35127
--webserver_interface=127.1.192.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab
--principal=kudu/127.1.192.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:40417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.192.254:35127
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:02:23.489108 1817 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:02:23.489511 1817 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:02:23.489607 1817 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:02:23.493826 1817 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:02:23.493939 1817 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:02:23.493975 1817 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:02:23.494004 1817 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:02:23.494040 1817 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:02:23.499915 1817 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:40417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786240941968412-1795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786240941968412-1795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.192.254:35127
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab
--principal=kudu/127.1.192.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.192.254:35127
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.192.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1817
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786240941968412-1795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:02:23.501624 1817 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:02:23.502970 1817 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:02:23.516906 1825 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:02:23.516991 1817 server_base.cc:1034] running on GCE node
W20260809 02:02:23.517267 1823 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:02:23.516991 1822 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:02:23.518338 1817 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:02:23.520290 1817 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:02:23.521548 1817 hybrid_clock.cc:648] HybridClock initialized: now 1786240943521485 us; error 97 us; skew 500 ppm
Aug 09 02:02:23 dist-test-slave-ss4n krb5kdc[1804](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240943, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:02:23.525146 1817 init.cc:377] Logged in from keytab as kudu/127.1.192.254@KRBTEST.COM (short username kudu)
I20260809 02:02:23.526468 1817 webserver.cc:460] Webserver started at http://127.1.192.254:37795/ using document root <none> and password file <none>
I20260809 02:02:23.527464 1817 fs_manager.cc:359] Metadata directory not provided
I20260809 02:02:23.527539 1817 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:02:23.528020 1817 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:02:23.531208 1817 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786240941968412-1795-0/minicluster-data/master-0/data/instance:
uuid: "03def2b9c05749c4a98d66edf122a3b6"
format_stamp: "Formatted at 2026-08-09 02:02:23 on dist-test-slave-ss4n"
server_key: "9df4bea9abb1dffdd105d5877f6364f8"
server_key_iv: "d6fd01bc30235cd236c52616d035ec55"
server_key_version: "encryptionkey@0"
I20260809 02:02:23.531996 1817 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786240941968412-1795-0/minicluster-data/master-0/wal/instance:
uuid: "03def2b9c05749c4a98d66edf122a3b6"
format_stamp: "Formatted at 2026-08-09 02:02:23 on dist-test-slave-ss4n"
server_key: "9df4bea9abb1dffdd105d5877f6364f8"
server_key_iv: "d6fd01bc30235cd236c52616d035ec55"
server_key_version: "encryptionkey@0"
I20260809 02:02:23.538481 1817 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260809 02:02:23.543018 1832 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:23.545126 1817 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260809 02:02:23.545394 1817 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786240941968412-1795-0/minicluster-data/master-0/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786240941968412-1795-0/minicluster-data/master-0/wal
uuid: "03def2b9c05749c4a98d66edf122a3b6"
format_stamp: "Formatted at 2026-08-09 02:02:23 on dist-test-slave-ss4n"
server_key: "9df4bea9abb1dffdd105d5877f6364f8"
server_key_iv: "d6fd01bc30235cd236c52616d035ec55"
server_key_version: "encryptionkey@0"
I20260809 02:02:23.545640 1817 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786240941968412-1795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786240941968412-1795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786240941968412-1795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:02:23.568192 1817 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:02:23.572614 1817 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:02:23.572892 1817 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:02:23.587486 1817 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.254:35127
I20260809 02:02:23.587626 1890 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.254:35127 every 8 connection(s)
I20260809 02:02:23.588989 1817 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
I20260809 02:02:23.591691 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 1817
I20260809 02:02:23.591907 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786240941968412-1795-0/minicluster-data/master-0/wal/instance
I20260809 02:02:23.593351 1795 external_mini_cluster.cc:1428] Setting key b7de9483819bf5d7fb2fffad55494ed2
I20260809 02:02:23.594835 1891 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:02:23.607317 1891 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03def2b9c05749c4a98d66edf122a3b6: Bootstrap starting.
I20260809 02:02:23.612507 1891 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 03def2b9c05749c4a98d66edf122a3b6: Neither blocks nor log segments found. Creating new log.
I20260809 02:02:23.614007 1891 log.cc:824] T 00000000000000000000000000000000 P 03def2b9c05749c4a98d66edf122a3b6: Log is configured to *not* fsync() on all Append() calls
I20260809 02:02:23.617947 1891 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03def2b9c05749c4a98d66edf122a3b6: No bootstrap required, opened a new log
Aug 09 02:02:23 dist-test-slave-ss4n krb5kdc[1804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240943, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.192.254@KRBTEST.COM
I20260809 02:02:23.624730 1891 raft_consensus.cc:348] T 00000000000000000000000000000000 P 03def2b9c05749c4a98d66edf122a3b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03def2b9c05749c4a98d66edf122a3b6" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 35127 } }
I20260809 02:02:23.625059 1891 raft_consensus.cc:374] T 00000000000000000000000000000000 P 03def2b9c05749c4a98d66edf122a3b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:02:23.625174 1891 raft_consensus.cc:729] T 00000000000000000000000000000000 P 03def2b9c05749c4a98d66edf122a3b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03def2b9c05749c4a98d66edf122a3b6, State: Initialized, Role: FOLLOWER
I20260809 02:02:23.626056 1891 consensus_queue.cc:260] T 00000000000000000000000000000000 P 03def2b9c05749c4a98d66edf122a3b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03def2b9c05749c4a98d66edf122a3b6" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 35127 } }
I20260809 02:02:23.626262 1891 raft_consensus.cc:388] T 00000000000000000000000000000000 P 03def2b9c05749c4a98d66edf122a3b6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:02:23.626415 1891 raft_consensus.cc:482] T 00000000000000000000000000000000 P 03def2b9c05749c4a98d66edf122a3b6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:02:23.626569 1891 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 03def2b9c05749c4a98d66edf122a3b6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:02:23.627672 1891 raft_consensus.cc:504] T 00000000000000000000000000000000 P 03def2b9c05749c4a98d66edf122a3b6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03def2b9c05749c4a98d66edf122a3b6" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 35127 } }
I20260809 02:02:23.628293 1891 leader_election.cc:302] T 00000000000000000000000000000000 P 03def2b9c05749c4a98d66edf122a3b6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 03def2b9c05749c4a98d66edf122a3b6; no voters:
I20260809 02:02:23.628786 1891 leader_election.cc:288] T 00000000000000000000000000000000 P 03def2b9c05749c4a98d66edf122a3b6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:02:23.629247 1896 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 03def2b9c05749c4a98d66edf122a3b6 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:02:23.629528 1896 raft_consensus.cc:686] T 00000000000000000000000000000000 P 03def2b9c05749c4a98d66edf122a3b6 [term 1 LEADER]: Becoming Leader. State: Replica: 03def2b9c05749c4a98d66edf122a3b6, State: Running, Role: LEADER
I20260809 02:02:23.629807 1894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:23.600253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33042 (local address 127.1.192.254:35127)
0809 02:02:23.600772 (+ 519us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:23.600786 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:23.600822 (+ 36us) server_negotiation.cc:407] Connection header received
0809 02:02:23.606024 (+ 5202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:23.606044 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:23.606489 (+ 445us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:23.606823 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:23.611357 (+ 4534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:23.612551 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:23.614871 (+ 2320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:23.615676 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:23.615809 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:23.622052 (+ 6243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:02:23.622098 (+ 46us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:02:23.622116 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:02:23.622182 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:02:23.624618 (+ 2436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:23.625440 (+ 822us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:23.625446 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:23.625450 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:23.625714 (+ 264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:23.626124 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:23.626128 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:23.626130 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:23.626933 (+ 803us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:02:23.627260 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:23.627951 (+ 691us) server_negotiation.cc:299] Negotiation successful
0809 02:02:23.628303 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":397,"thread_start_us":146,"threads_started":1}
I20260809 02:02:23.629974 1896 consensus_queue.cc:237] T 00000000000000000000000000000000 P 03def2b9c05749c4a98d66edf122a3b6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03def2b9c05749c4a98d66edf122a3b6" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 35127 } }
I20260809 02:02:23.630807 1891 sys_catalog.cc:564] T 00000000000000000000000000000000 P 03def2b9c05749c4a98d66edf122a3b6 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:02:23.632154 1897 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03def2b9c05749c4a98d66edf122a3b6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 03def2b9c05749c4a98d66edf122a3b6. Latest consensus state: current_term: 1 leader_uuid: "03def2b9c05749c4a98d66edf122a3b6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03def2b9c05749c4a98d66edf122a3b6" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 35127 } } }
I20260809 02:02:23.632313 1897 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03def2b9c05749c4a98d66edf122a3b6 [sys.catalog]: This master's current role is: LEADER
I20260809 02:02:23.633677 1901 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:02:23.636308 1898 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03def2b9c05749c4a98d66edf122a3b6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "03def2b9c05749c4a98d66edf122a3b6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03def2b9c05749c4a98d66edf122a3b6" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 35127 } } }
I20260809 02:02:23.636441 1898 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03def2b9c05749c4a98d66edf122a3b6 [sys.catalog]: This master's current role is: LEADER
I20260809 02:02:23.639570 1901 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:02:23.649530 1901 catalog_manager.cc:1347] Generated new cluster ID: 4edf4c3e1cac426aae39226032f2b806
I20260809 02:02:23.649708 1901 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:02:23.666550 1901 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:02:23.667663 1901 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:02:23.677373 1901 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 03def2b9c05749c4a98d66edf122a3b6: Generated new TSK 0
I20260809 02:02:23.678598 1901 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.192.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.192.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.192.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.193.keytab.
Entry for principal kudu/127.1.192.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.193.keytab.
I20260809 02:02:23.725772 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786240941968412-1795-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786240941968412-1795-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786240941968412-1795-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786240941968412-1795-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.192.193:0
--local_ip_for_outbound_sockets=127.1.192.193
--webserver_interface=127.1.192.193
--webserver_port=0
--tserver_master_addrs=127.1.192.254:35127
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.193.keytab
--principal=kudu/127.1.192.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:40417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:02:23.877550 1918 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:02:23.877830 1918 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:02:23.877876 1918 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:02:23.882251 1918 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:02:23.882321 1918 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:02:23.882385 1918 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.192.193
I20260809 02:02:23.888053 1918 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:40417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786240941968412-1795-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786240941968412-1795-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.193.keytab
--principal=kudu/127.1.192.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.192.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786240941968412-1795-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.192.193
--webserver_port=0
--tserver_master_addrs=127.1.192.254:35127
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1918
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.192.193
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786240941968412-1795-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:02:23.889222 1918 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:02:23.890439 1918 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:02:23.901872 1923 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:02:23.902027 1918 server_base.cc:1034] running on GCE node
W20260809 02:02:23.901875 1925 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:02:23.902247 1927 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:02:23.902702 1918 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:02:23.903457 1918 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:02:23.904713 1918 hybrid_clock.cc:648] HybridClock initialized: now 1786240943904698 us; error 30 us; skew 500 ppm
Aug 09 02:02:23 dist-test-slave-ss4n krb5kdc[1804](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240943, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:02:23.913136 1918 init.cc:377] Logged in from keytab as kudu/127.1.192.193@KRBTEST.COM (short username kudu)
I20260809 02:02:23.914788 1918 webserver.cc:460] Webserver started at http://127.1.192.193:35881/ using document root <none> and password file <none>
I20260809 02:02:23.915689 1918 fs_manager.cc:359] Metadata directory not provided
I20260809 02:02:23.915802 1918 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:02:23.916098 1918 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:02:23.919473 1918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786240941968412-1795-0/minicluster-data/ts-0/data/instance:
uuid: "b8ed27af3bf643e6945f1d8598536d4a"
format_stamp: "Formatted at 2026-08-09 02:02:23 on dist-test-slave-ss4n"
server_key: "c30fe7322eb4cf364c5884c8ec3e8073"
server_key_iv: "487a677c0defc51ec3e08caf6047c613"
server_key_version: "encryptionkey@0"
I20260809 02:02:23.920380 1918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786240941968412-1795-0/minicluster-data/ts-0/wal/instance:
uuid: "b8ed27af3bf643e6945f1d8598536d4a"
format_stamp: "Formatted at 2026-08-09 02:02:23 on dist-test-slave-ss4n"
server_key: "c30fe7322eb4cf364c5884c8ec3e8073"
server_key_iv: "487a677c0defc51ec3e08caf6047c613"
server_key_version: "encryptionkey@0"
I20260809 02:02:23.925945 1918 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260809 02:02:23.928823 1935 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:23.930318 1918 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 02:02:23.930434 1918 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786240941968412-1795-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786240941968412-1795-0/minicluster-data/ts-0/wal
uuid: "b8ed27af3bf643e6945f1d8598536d4a"
format_stamp: "Formatted at 2026-08-09 02:02:23 on dist-test-slave-ss4n"
server_key: "c30fe7322eb4cf364c5884c8ec3e8073"
server_key_iv: "487a677c0defc51ec3e08caf6047c613"
server_key_version: "encryptionkey@0"
I20260809 02:02:23.930541 1918 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786240941968412-1795-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786240941968412-1795-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786240941968412-1795-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:02:23.965442 1918 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:02:23.969017 1918 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:02:23.969193 1918 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:02:23.970002 1918 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:02:23.971216 1918 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:02:23.971274 1918 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:23.971320 1918 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:02:23.971338 1918 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:23.983284 1918 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.193:46879
I20260809 02:02:23.983309 2056 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.193:46879 every 8 connection(s)
I20260809 02:02:23.984368 1918 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786240941968412-1795-0/minicluster-data/ts-0/data/info.pb
I20260809 02:02:23.986570 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 1918
I20260809 02:02:23.986670 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786240941968412-1795-0/minicluster-data/ts-0/wal/instance
I20260809 02:02:23.987006 1795 external_mini_cluster.cc:1428] Setting key e925cd18049ee51c6672aee2c614aa59
Aug 09 02:02:23 dist-test-slave-ss4n krb5kdc[1804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240943, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.193@KRBTEST.COM for kudu/127.1.192.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.192.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.192.194@KRBTEST.COM" created.
I20260809 02:02:24.001605 1894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:23.987129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.193:47359 (local address 127.1.192.254:35127)
0809 02:02:23.987287 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:23.987293 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:23.988347 (+ 1054us) server_negotiation.cc:407] Connection header received
0809 02:02:23.990145 (+ 1798us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:23.990151 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:23.990219 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:23.990334 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:23.992144 (+ 1810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:23.992741 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:23.994104 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:23.994368 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:23.994500 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:23.997429 (+ 2929us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:02:23.997433 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:02:23.997436 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:02:23.997465 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:02:23.999072 (+ 1607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:23.999622 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:23.999625 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:23.999626 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:23.999719 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:24.000106 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:24.000109 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:24.000111 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:24.000398 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:02:24.000513 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:24.001138 (+ 625us) server_negotiation.cc:299] Negotiation successful
0809 02:02:24.001323 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 02:02:24.002723 2059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:23.987579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:35127 (local address 127.1.192.193:47359)
0809 02:02:23.988186 (+ 607us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:23.988244 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0809 02:02:23.989908 (+ 1664us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:02:23.990551 (+ 643us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:02:23.990563 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:02:23.991175 (+ 612us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:02:23.991955 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:23.991978 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:23.992892 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:23.992896 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:23.993961 (+ 1065us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:23.993972 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:23.994549 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:23.994553 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:23.994736 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:23.995414 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:02:23.995432 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:02:23.997235 (+ 1803us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:02:23.999197 (+ 1962us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:23.999202 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:23.999216 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:23.999499 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:23.999814 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:23.999816 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:23.999818 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:23.999966 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:24.000518 (+ 552us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:02:24.000523 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:02:24.000792 (+ 269us) client_negotiation.cc:769] Sending connection context
0809 02:02:24.001025 (+ 233us) client_negotiation.cc:240] Negotiation successful
0809 02:02:24.001377 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":162,"threads_started":1}
I20260809 02:02:24.004140 2057 heartbeater.cc:347] Connected to a master server at 127.1.192.254:35127
I20260809 02:02:24.004426 2057 heartbeater.cc:464] Registering TS with master...
I20260809 02:02:24.005038 2057 heartbeater.cc:511] Master 127.1.192.254:35127 requested a full tablet report, sending...
I20260809 02:02:24.007341 1850 ts_manager.cc:194] Registered new tserver with Master: b8ed27af3bf643e6945f1d8598536d4a (127.1.192.193:46879)
I20260809 02:02:24.008930 1850 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.192.193@KRBTEST.COM'} at 127.1.192.193:47359
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.192.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.194.keytab.
Entry for principal kudu/127.1.192.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.194.keytab.
I20260809 02:02:24.022266 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786240941968412-1795-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786240941968412-1795-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786240941968412-1795-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786240941968412-1795-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.192.194:0
--local_ip_for_outbound_sockets=127.1.192.194
--webserver_interface=127.1.192.194
--webserver_port=0
--tserver_master_addrs=127.1.192.254:35127
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.194.keytab
--principal=kudu/127.1.192.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:40417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:02:24.141413 2062 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:02:24.141742 2062 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:02:24.141789 2062 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:02:24.145951 2062 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:02:24.146034 2062 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:02:24.146099 2062 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.192.194
I20260809 02:02:24.151816 2062 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:40417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786240941968412-1795-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786240941968412-1795-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.194.keytab
--principal=kudu/127.1.192.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.192.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786240941968412-1795-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.192.194
--webserver_port=0
--tserver_master_addrs=127.1.192.254:35127
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2062
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.192.194
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786240941968412-1795-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:02:24.152871 2062 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:02:24.153940 2062 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:02:24.163806 2071 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:02:24.163832 2068 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:02:24.163806 2067 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:02:24.164067 2062 server_base.cc:1034] running on GCE node
I20260809 02:02:24.164572 2062 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:02:24.165297 2062 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:02:24.166484 2062 hybrid_clock.cc:648] HybridClock initialized: now 1786240944166463 us; error 44 us; skew 500 ppm
Aug 09 02:02:24 dist-test-slave-ss4n krb5kdc[1804](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240944, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:02:24.169340 2062 init.cc:377] Logged in from keytab as kudu/127.1.192.194@KRBTEST.COM (short username kudu)
I20260809 02:02:24.170336 2062 webserver.cc:460] Webserver started at http://127.1.192.194:44891/ using document root <none> and password file <none>
I20260809 02:02:24.170925 2062 fs_manager.cc:359] Metadata directory not provided
I20260809 02:02:24.171020 2062 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:02:24.171264 2062 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:02:24.173328 2062 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786240941968412-1795-0/minicluster-data/ts-1/data/instance:
uuid: "1790e6e810c349228985308acef2153e"
format_stamp: "Formatted at 2026-08-09 02:02:24 on dist-test-slave-ss4n"
server_key: "90471dd858cf590ba745188b77b0e243"
server_key_iv: "7d597d32c5e8e86431e3525c455e979f"
server_key_version: "encryptionkey@0"
I20260809 02:02:24.174072 2062 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786240941968412-1795-0/minicluster-data/ts-1/wal/instance:
uuid: "1790e6e810c349228985308acef2153e"
format_stamp: "Formatted at 2026-08-09 02:02:24 on dist-test-slave-ss4n"
server_key: "90471dd858cf590ba745188b77b0e243"
server_key_iv: "7d597d32c5e8e86431e3525c455e979f"
server_key_version: "encryptionkey@0"
I20260809 02:02:24.178125 2062 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 02:02:24.180647 2079 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:24.182089 2062 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 02:02:24.182250 2062 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786240941968412-1795-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786240941968412-1795-0/minicluster-data/ts-1/wal
uuid: "1790e6e810c349228985308acef2153e"
format_stamp: "Formatted at 2026-08-09 02:02:24 on dist-test-slave-ss4n"
server_key: "90471dd858cf590ba745188b77b0e243"
server_key_iv: "7d597d32c5e8e86431e3525c455e979f"
server_key_version: "encryptionkey@0"
I20260809 02:02:24.182381 2062 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786240941968412-1795-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786240941968412-1795-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786240941968412-1795-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:02:24.201246 2062 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:02:24.204842 2062 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:02:24.205108 2062 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:02:24.205828 2062 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:02:24.206768 2062 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:02:24.206849 2062 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:24.206926 2062 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:02:24.206986 2062 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:24.217669 2062 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.194:42393
I20260809 02:02:24.217749 2200 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.194:42393 every 8 connection(s)
I20260809 02:02:24.218722 2062 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786240941968412-1795-0/minicluster-data/ts-1/data/info.pb
I20260809 02:02:24.220386 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 2062
I20260809 02:02:24.220611 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786240941968412-1795-0/minicluster-data/ts-1/wal/instance
I20260809 02:02:24.220971 1795 external_mini_cluster.cc:1428] Setting key ba6d37f272e573218d6f32a15d9ac869
Aug 09 02:02:24 dist-test-slave-ss4n krb5kdc[1804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240944, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.194@KRBTEST.COM for kudu/127.1.192.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.192.195@KRBTEST.COM; defaulting to no policy
I20260809 02:02:24.235074 1894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:24.220990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.194:45209 (local address 127.1.192.254:35127)
0809 02:02:24.221166 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:24.221171 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:24.222200 (+ 1029us) server_negotiation.cc:407] Connection header received
0809 02:02:24.223693 (+ 1493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:24.223697 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:24.223749 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:24.223825 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:24.225051 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:24.225965 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:24.227237 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:24.227680 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:24.227814 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:24.230795 (+ 2981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:02:24.230800 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:02:24.230803 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:02:24.230835 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:02:24.232490 (+ 1655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:24.233134 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:24.233139 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:24.233141 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:24.233229 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:24.233675 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:24.233680 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:24.233682 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:24.233893 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:02:24.234014 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:24.234680 (+ 666us) server_negotiation.cc:299] Negotiation successful
0809 02:02:24.234821 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.192.195@KRBTEST.COM" created.
I20260809 02:02:24.236246 2203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:24.221393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:35127 (local address 127.1.192.194:45209)
0809 02:02:24.222034 (+ 641us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:24.222079 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 02:02:24.223472 (+ 1393us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:02:24.223994 (+ 522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:02:24.224002 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:02:24.224356 (+ 354us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:02:24.224839 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:24.224852 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:24.226121 (+ 1269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:24.226124 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:24.227094 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:24.227104 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:24.227862 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:24.227865 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:24.228051 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:24.228733 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:02:24.228751 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:02:24.230583 (+ 1832us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:02:24.232636 (+ 2053us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:24.232643 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:24.232659 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:24.233020 (+ 361us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:24.233336 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:24.233340 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:24.233342 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:24.233559 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:24.234033 (+ 474us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:02:24.234042 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:02:24.234350 (+ 308us) client_negotiation.cc:769] Sending connection context
0809 02:02:24.234614 (+ 264us) client_negotiation.cc:240] Negotiation successful
0809 02:02:24.235057 (+ 443us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":176,"threads_started":1}
I20260809 02:02:24.237592 2201 heartbeater.cc:347] Connected to a master server at 127.1.192.254:35127
I20260809 02:02:24.237897 2201 heartbeater.cc:464] Registering TS with master...
I20260809 02:02:24.238504 2201 heartbeater.cc:511] Master 127.1.192.254:35127 requested a full tablet report, sending...
I20260809 02:02:24.239758 1850 ts_manager.cc:194] Registered new tserver with Master: 1790e6e810c349228985308acef2153e (127.1.192.194:42393)
I20260809 02:02:24.240435 1850 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.192.194@KRBTEST.COM'} at 127.1.192.194:45209
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.192.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.195.keytab.
Entry for principal kudu/127.1.192.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.195.keytab.
I20260809 02:02:24.255002 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786240941968412-1795-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786240941968412-1795-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786240941968412-1795-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786240941968412-1795-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.192.195:0
--local_ip_for_outbound_sockets=127.1.192.195
--webserver_interface=127.1.192.195
--webserver_port=0
--tserver_master_addrs=127.1.192.254:35127
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.195.keytab
--principal=kudu/127.1.192.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:40417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:02:24.372412 2207 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:02:24.372720 2207 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:02:24.372768 2207 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:02:24.376857 2207 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:02:24.376929 2207 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:02:24.376992 2207 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.192.195
I20260809 02:02:24.382854 2207 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:40417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786240941968412-1795-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786240941968412-1795-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.195.keytab
--principal=kudu/127.1.192.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.192.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786240941968412-1795-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.192.195
--webserver_port=0
--tserver_master_addrs=127.1.192.254:35127
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2207
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.192.195
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786240941968412-1795-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:02:24.383991 2207 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:02:24.385017 2207 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:02:24.396243 2212 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:02:24.396310 2207 server_base.cc:1034] running on GCE node
W20260809 02:02:24.396243 2215 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:02:24.396236 2213 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:02:24.396976 2207 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:02:24.397783 2207 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:02:24.398979 2207 hybrid_clock.cc:648] HybridClock initialized: now 1786240944398941 us; error 52 us; skew 500 ppm
Aug 09 02:02:24 dist-test-slave-ss4n krb5kdc[1804](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240944, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:02:24.402186 2207 init.cc:377] Logged in from keytab as kudu/127.1.192.195@KRBTEST.COM (short username kudu)
I20260809 02:02:24.403189 2207 webserver.cc:460] Webserver started at http://127.1.192.195:44767/ using document root <none> and password file <none>
I20260809 02:02:24.403728 2207 fs_manager.cc:359] Metadata directory not provided
I20260809 02:02:24.403791 2207 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:02:24.403957 2207 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:02:24.406100 2207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786240941968412-1795-0/minicluster-data/ts-2/data/instance:
uuid: "ddfba6823f8b4d10ab0816248a87a201"
format_stamp: "Formatted at 2026-08-09 02:02:24 on dist-test-slave-ss4n"
server_key: "43ae4433fb6047ebff0f107769492883"
server_key_iv: "17b0a284419c84b0836ee864ebb20b7a"
server_key_version: "encryptionkey@0"
I20260809 02:02:24.406658 2207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786240941968412-1795-0/minicluster-data/ts-2/wal/instance:
uuid: "ddfba6823f8b4d10ab0816248a87a201"
format_stamp: "Formatted at 2026-08-09 02:02:24 on dist-test-slave-ss4n"
server_key: "43ae4433fb6047ebff0f107769492883"
server_key_iv: "17b0a284419c84b0836ee864ebb20b7a"
server_key_version: "encryptionkey@0"
I20260809 02:02:24.410488 2207 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260809 02:02:24.412954 2223 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:24.414336 2207 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 02:02:24.414443 2207 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786240941968412-1795-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786240941968412-1795-0/minicluster-data/ts-2/wal
uuid: "ddfba6823f8b4d10ab0816248a87a201"
format_stamp: "Formatted at 2026-08-09 02:02:24 on dist-test-slave-ss4n"
server_key: "43ae4433fb6047ebff0f107769492883"
server_key_iv: "17b0a284419c84b0836ee864ebb20b7a"
server_key_version: "encryptionkey@0"
I20260809 02:02:24.414577 2207 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786240941968412-1795-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786240941968412-1795-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786240941968412-1795-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:02:24.426872 2207 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:02:24.430761 2207 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:02:24.430999 2207 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:02:24.431681 2207 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:02:24.432642 2207 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:02:24.432717 2207 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:24.432778 2207 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:02:24.432809 2207 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:24.443538 2207 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.195:39367
I20260809 02:02:24.443568 2345 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.195:39367 every 8 connection(s)
I20260809 02:02:24.444499 2207 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786240941968412-1795-0/minicluster-data/ts-2/data/info.pb
I20260809 02:02:24.453012 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 2207
I20260809 02:02:24.453153 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786240941968412-1795-0/minicluster-data/ts-2/wal/instance
I20260809 02:02:24.453449 1795 external_mini_cluster.cc:1428] Setting key 69846e19d14a6dc1d5253a5d436302a9
Aug 09 02:02:24 dist-test-slave-ss4n krb5kdc[1804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240944, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.195@KRBTEST.COM for kudu/127.1.192.254@KRBTEST.COM
I20260809 02:02:24.459692 1894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:24.446648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.195:34007 (local address 127.1.192.254:35127)
0809 02:02:24.446821 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:24.446825 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:24.447701 (+ 876us) server_negotiation.cc:407] Connection header received
0809 02:02:24.449176 (+ 1475us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:24.449179 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:24.449227 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:24.449304 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:24.450619 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:24.451198 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:24.452381 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:24.452596 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:24.452678 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:24.455987 (+ 3309us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:02:24.455990 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:02:24.455992 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:02:24.456017 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:02:24.457727 (+ 1710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:24.458249 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:24.458251 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:24.458252 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:24.458326 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:24.458673 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:24.458675 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:24.458677 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:24.458855 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:02:24.458932 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:24.459455 (+ 523us) server_negotiation.cc:299] Negotiation successful
0809 02:02:24.459560 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260809 02:02:24.460578 2348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:24.447070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:35127 (local address 127.1.192.195:34007)
0809 02:02:24.447532 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:24.447582 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 02:02:24.448824 (+ 1242us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:02:24.449566 (+ 742us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:02:24.449573 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:02:24.449920 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:02:24.450479 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:24.450494 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:24.451372 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:24.451375 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:24.452276 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:24.452284 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:24.452756 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:24.452761 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:24.452966 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:24.453789 (+ 823us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:02:24.453810 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:02:24.455791 (+ 1981us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:02:24.457841 (+ 2050us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:24.457846 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:24.457858 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:24.458155 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:24.458404 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:24.458406 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:24.458407 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:24.458567 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:24.458956 (+ 389us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:02:24.458970 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:02:24.459243 (+ 273us) client_negotiation.cc:769] Sending connection context
0809 02:02:24.459407 (+ 164us) client_negotiation.cc:240] Negotiation successful
0809 02:02:24.459691 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":128,"threads_started":1}
I20260809 02:02:24.461844 2346 heartbeater.cc:347] Connected to a master server at 127.1.192.254:35127
I20260809 02:02:24.462085 2346 heartbeater.cc:464] Registering TS with master...
I20260809 02:02:24.462666 2346 heartbeater.cc:511] Master 127.1.192.254:35127 requested a full tablet report, sending...
I20260809 02:02:24.463748 1850 ts_manager.cc:194] Registered new tserver with Master: ddfba6823f8b4d10ab0816248a87a201 (127.1.192.195:39367)
I20260809 02:02:24.464326 1850 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.192.195@KRBTEST.COM'} at 127.1.192.195:34007
I20260809 02:02:24.468726 1795 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 02:02:24 dist-test-slave-ss4n krb5kdc[1804](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240944, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 02:02:24 dist-test-slave-ss4n krb5kdc[1804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240944, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.192.193@KRBTEST.COM
I20260809 02:02:24.499604 2353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:24.488682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44980 (local address 127.1.192.193:46879)
0809 02:02:24.488976 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:24.488997 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:24.489127 (+ 130us) server_negotiation.cc:407] Connection header received
0809 02:02:24.489307 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:24.489313 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:24.489571 (+ 258us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:24.489763 (+ 192us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:24.490548 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:24.491433 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:24.492384 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:24.492816 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:24.492973 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:24.495763 (+ 2790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:02:24.495781 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:02:24.495796 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:02:24.495823 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:02:24.497571 (+ 1748us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:24.498066 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:24.498073 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:24.498075 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:24.498175 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:24.498498 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:24.498502 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:24.498504 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:24.498780 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:02:24.499059 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:24.499224 (+ 165us) server_negotiation.cc:299] Negotiation successful
0809 02:02:24.499397 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":72,"threads_started":1}
W20260809 02:02:24.500734 1966 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44980
Aug 09 02:02:24 dist-test-slave-ss4n krb5kdc[1804](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240944, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 02:02:24 dist-test-slave-ss4n krb5kdc[1804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240944, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.192.193@KRBTEST.COM
I20260809 02:02:24.529656 2353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:24.520657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44996 (local address 127.1.192.193:46879)
0809 02:02:24.520918 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:24.520925 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:24.521055 (+ 130us) server_negotiation.cc:407] Connection header received
0809 02:02:24.521234 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:24.521243 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:24.521296 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:24.521391 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:24.522188 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:24.522752 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:24.523620 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:24.523865 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:24.523986 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:24.526572 (+ 2586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:02:24.526576 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:02:24.526578 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:02:24.526604 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:02:24.527946 (+ 1342us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:24.528375 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:24.528379 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:24.528381 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:24.528465 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:24.528753 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:24.528757 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:24.528759 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:24.528941 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:02:24.529080 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:24.529324 (+ 244us) server_negotiation.cc:299] Negotiation successful
0809 02:02:24.529452 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163}
I20260809 02:02:24.530964 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 1918
I20260809 02:02:24.539184 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 2062
I20260809 02:02:24.546371 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 2207
I20260809 02:02:24.553395 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 1817
2026-08-09T02:02:24Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2548 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786240941968412-1795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:02:24 dist-test-slave-ss4n krb5kdc[2363](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:02:24 dist-test-slave-ss4n krb5kdc[2363](info): set up 2 sockets
Aug 09 02:02:24 dist-test-slave-ss4n krb5kdc[2363](info): commencing operation
krb5kdc: starting...
W20260809 02:02:25.651470 1795 mini_kdc.cc:121] Time spent starting KDC: real 1.072s user 0.006s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786240941968412-1795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786240941968412-1795-0/krb5kdc/test-user.keytab.
Aug 09 02:02:25 dist-test-slave-ss4n krb5kdc[2363](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240945, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:02:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:02:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.192.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.192.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.192.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab.
Entry for principal kudu/127.1.192.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab.
I20260809 02:02:25.789726 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786240941968412-1795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786240941968412-1795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786240941968412-1795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.192.254:34951
--webserver_interface=127.1.192.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab
--principal=kudu/127.1.192.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:41551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.192.254:34951
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:02:25.905299 2377 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:02:25.905748 2377 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:02:25.905841 2377 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:02:25.909965 2377 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:02:25.910084 2377 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:02:25.910120 2377 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:02:25.910161 2377 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:02:25.910197 2377 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:02:25.918383 2377 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:41551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786240941968412-1795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786240941968412-1795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.192.254:34951
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab
--principal=kudu/127.1.192.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.192.254:34951
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.192.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2377
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786240941968412-1795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:02:25.920015 2377 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:02:25.921319 2377 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:02:25.930732 2383 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:02:25.930734 2382 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:02:25.930732 2385 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:02:25.931038 2377 server_base.cc:1034] running on GCE node
I20260809 02:02:25.931656 2377 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:02:25.932857 2377 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:02:25.934055 2377 hybrid_clock.cc:648] HybridClock initialized: now 1786240945934037 us; error 35 us; skew 500 ppm
Aug 09 02:02:25 dist-test-slave-ss4n krb5kdc[2363](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240945, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:02:25.937189 2377 init.cc:377] Logged in from keytab as kudu/127.1.192.254@KRBTEST.COM (short username kudu)
I20260809 02:02:25.938241 2377 webserver.cc:460] Webserver started at http://127.1.192.254:37945/ using document root <none> and password file <none>
I20260809 02:02:25.938836 2377 fs_manager.cc:359] Metadata directory not provided
I20260809 02:02:25.938935 2377 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:02:25.939179 2377 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:02:25.941359 2377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786240941968412-1795-0/minicluster-data/master-0/data/instance:
uuid: "097fff488ff74b34bc1ed4bc9c23aae1"
format_stamp: "Formatted at 2026-08-09 02:02:25 on dist-test-slave-ss4n"
server_key: "de6109c0636686f8b643a4f1752cff8d"
server_key_iv: "b2745e00d691012ba3415f6caba5ab8a"
server_key_version: "encryptionkey@0"
I20260809 02:02:25.942026 2377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786240941968412-1795-0/minicluster-data/master-0/wal/instance:
uuid: "097fff488ff74b34bc1ed4bc9c23aae1"
format_stamp: "Formatted at 2026-08-09 02:02:25 on dist-test-slave-ss4n"
server_key: "de6109c0636686f8b643a4f1752cff8d"
server_key_iv: "b2745e00d691012ba3415f6caba5ab8a"
server_key_version: "encryptionkey@0"
I20260809 02:02:25.946110 2377 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 02:02:25.948814 2392 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:25.950356 2377 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 02:02:25.950542 2377 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786240941968412-1795-0/minicluster-data/master-0/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786240941968412-1795-0/minicluster-data/master-0/wal
uuid: "097fff488ff74b34bc1ed4bc9c23aae1"
format_stamp: "Formatted at 2026-08-09 02:02:25 on dist-test-slave-ss4n"
server_key: "de6109c0636686f8b643a4f1752cff8d"
server_key_iv: "b2745e00d691012ba3415f6caba5ab8a"
server_key_version: "encryptionkey@0"
I20260809 02:02:25.950690 2377 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786240941968412-1795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786240941968412-1795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786240941968412-1795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:02:25.967070 2377 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:02:25.970196 2377 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:02:25.970420 2377 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:02:25.977782 2377 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.254:34951
I20260809 02:02:25.977810 2448 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.254:34951 every 8 connection(s)
I20260809 02:02:25.978791 2377 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
I20260809 02:02:25.981596 2449 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:02:25.987411 2449 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 097fff488ff74b34bc1ed4bc9c23aae1: Bootstrap starting.
I20260809 02:02:25.987808 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 2377
I20260809 02:02:25.987901 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786240941968412-1795-0/minicluster-data/master-0/wal/instance
I20260809 02:02:25.988169 1795 external_mini_cluster.cc:1428] Setting key f44b23ea494cacd29c698edb5f06d5a7
I20260809 02:02:25.990163 2449 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 097fff488ff74b34bc1ed4bc9c23aae1: Neither blocks nor log segments found. Creating new log.
I20260809 02:02:25.990919 2449 log.cc:824] T 00000000000000000000000000000000 P 097fff488ff74b34bc1ed4bc9c23aae1: Log is configured to *not* fsync() on all Append() calls
I20260809 02:02:25.992779 2449 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 097fff488ff74b34bc1ed4bc9c23aae1: No bootstrap required, opened a new log
I20260809 02:02:25.996575 2449 raft_consensus.cc:348] T 00000000000000000000000000000000 P 097fff488ff74b34bc1ed4bc9c23aae1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "097fff488ff74b34bc1ed4bc9c23aae1" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 34951 } }
I20260809 02:02:25.996838 2449 raft_consensus.cc:374] T 00000000000000000000000000000000 P 097fff488ff74b34bc1ed4bc9c23aae1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:02:25.996909 2449 raft_consensus.cc:729] T 00000000000000000000000000000000 P 097fff488ff74b34bc1ed4bc9c23aae1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 097fff488ff74b34bc1ed4bc9c23aae1, State: Initialized, Role: FOLLOWER
I20260809 02:02:25.997459 2449 consensus_queue.cc:260] T 00000000000000000000000000000000 P 097fff488ff74b34bc1ed4bc9c23aae1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "097fff488ff74b34bc1ed4bc9c23aae1" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 34951 } }
Aug 09 02:02:25 dist-test-slave-ss4n krb5kdc[2363](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240945, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.192.254@KRBTEST.COM
I20260809 02:02:25.997655 2449 raft_consensus.cc:388] T 00000000000000000000000000000000 P 097fff488ff74b34bc1ed4bc9c23aae1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:02:25.997740 2449 raft_consensus.cc:482] T 00000000000000000000000000000000 P 097fff488ff74b34bc1ed4bc9c23aae1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:02:25.997846 2449 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 097fff488ff74b34bc1ed4bc9c23aae1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:02:25.999029 2449 raft_consensus.cc:504] T 00000000000000000000000000000000 P 097fff488ff74b34bc1ed4bc9c23aae1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "097fff488ff74b34bc1ed4bc9c23aae1" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 34951 } }
I20260809 02:02:25.999396 2449 leader_election.cc:302] T 00000000000000000000000000000000 P 097fff488ff74b34bc1ed4bc9c23aae1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 097fff488ff74b34bc1ed4bc9c23aae1; no voters:
I20260809 02:02:25.999723 2449 leader_election.cc:288] T 00000000000000000000000000000000 P 097fff488ff74b34bc1ed4bc9c23aae1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:02:25.999848 2454 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 097fff488ff74b34bc1ed4bc9c23aae1 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:02:26.000056 2454 raft_consensus.cc:686] T 00000000000000000000000000000000 P 097fff488ff74b34bc1ed4bc9c23aae1 [term 1 LEADER]: Becoming Leader. State: Replica: 097fff488ff74b34bc1ed4bc9c23aae1, State: Running, Role: LEADER
I20260809 02:02:26.000833 2454 consensus_queue.cc:237] T 00000000000000000000000000000000 P 097fff488ff74b34bc1ed4bc9c23aae1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "097fff488ff74b34bc1ed4bc9c23aae1" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 34951 } }
I20260809 02:02:26.001024 2449 sys_catalog.cc:564] T 00000000000000000000000000000000 P 097fff488ff74b34bc1ed4bc9c23aae1 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:02:26.002709 2456 sys_catalog.cc:455] T 00000000000000000000000000000000 P 097fff488ff74b34bc1ed4bc9c23aae1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 097fff488ff74b34bc1ed4bc9c23aae1. Latest consensus state: current_term: 1 leader_uuid: "097fff488ff74b34bc1ed4bc9c23aae1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "097fff488ff74b34bc1ed4bc9c23aae1" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 34951 } } }
I20260809 02:02:26.002833 2456 sys_catalog.cc:458] T 00000000000000000000000000000000 P 097fff488ff74b34bc1ed4bc9c23aae1 [sys.catalog]: This master's current role is: LEADER
I20260809 02:02:26.003243 2455 sys_catalog.cc:455] T 00000000000000000000000000000000 P 097fff488ff74b34bc1ed4bc9c23aae1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "097fff488ff74b34bc1ed4bc9c23aae1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "097fff488ff74b34bc1ed4bc9c23aae1" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 34951 } } }
I20260809 02:02:26.003340 2455 sys_catalog.cc:458] T 00000000000000000000000000000000 P 097fff488ff74b34bc1ed4bc9c23aae1 [sys.catalog]: This master's current role is: LEADER
I20260809 02:02:26.003608 2467 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:02:26.004081 2452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:25.989593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37362 (local address 127.1.192.254:34951)
0809 02:02:25.990072 (+ 479us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:25.990087 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:25.990126 (+ 39us) server_negotiation.cc:407] Connection header received
0809 02:02:25.990866 (+ 740us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:25.990884 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:25.991258 (+ 374us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:25.991589 (+ 331us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:25.992390 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:25.993832 (+ 1442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:25.994747 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:25.995233 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:25.995426 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:25.998278 (+ 2852us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:02:25.998282 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:02:25.998295 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:02:25.998321 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:02:26.000547 (+ 2226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:26.001052 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:26.001058 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:26.001064 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:26.001215 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:26.001597 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:26.001601 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:26.001602 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:26.002123 (+ 521us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:02:26.002335 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:26.002674 (+ 339us) server_negotiation.cc:299] Negotiation successful
0809 02:02:26.002999 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":150,"threads_started":1}
I20260809 02:02:26.006788 2467 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:02:26.013036 2467 catalog_manager.cc:1347] Generated new cluster ID: dc3b55ded8ff421cabf11322b3d58f56
I20260809 02:02:26.013171 2467 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:02:26.048734 2467 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:02:26.049875 2467 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:02:26.062922 2467 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 097fff488ff74b34bc1ed4bc9c23aae1: Generated new TSK 0
I20260809 02:02:26.063738 2467 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.192.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.192.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.192.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.193.keytab.
Entry for principal kudu/127.1.192.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.193.keytab.
I20260809 02:02:26.105136 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786240941968412-1795-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786240941968412-1795-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786240941968412-1795-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786240941968412-1795-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.192.193:0
--local_ip_for_outbound_sockets=127.1.192.193
--webserver_interface=127.1.192.193
--webserver_port=0
--tserver_master_addrs=127.1.192.254:34951
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.193.keytab
--principal=kudu/127.1.192.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:41551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:02:26.219179 2476 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:02:26.219476 2476 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:02:26.219523 2476 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:02:26.223742 2476 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:02:26.223814 2476 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:02:26.223876 2476 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.192.193
I20260809 02:02:26.229192 2476 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:41551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786240941968412-1795-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786240941968412-1795-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.193.keytab
--principal=kudu/127.1.192.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.192.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786240941968412-1795-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.192.193
--webserver_port=0
--tserver_master_addrs=127.1.192.254:34951
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2476
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.192.193
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786240941968412-1795-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:02:26.230353 2476 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:02:26.231343 2476 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:02:26.241633 2484 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:02:26.241609 2481 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:02:26.241601 2482 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:02:26.241786 2476 server_base.cc:1034] running on GCE node
I20260809 02:02:26.242429 2476 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:02:26.243139 2476 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:02:26.244333 2476 hybrid_clock.cc:648] HybridClock initialized: now 1786240946244312 us; error 35 us; skew 500 ppm
Aug 09 02:02:26 dist-test-slave-ss4n krb5kdc[2363](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240946, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:02:26.247367 2476 init.cc:377] Logged in from keytab as kudu/127.1.192.193@KRBTEST.COM (short username kudu)
I20260809 02:02:26.248281 2476 webserver.cc:460] Webserver started at http://127.1.192.193:45187/ using document root <none> and password file <none>
I20260809 02:02:26.248904 2476 fs_manager.cc:359] Metadata directory not provided
I20260809 02:02:26.248970 2476 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:02:26.249199 2476 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:02:26.251410 2476 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786240941968412-1795-0/minicluster-data/ts-0/data/instance:
uuid: "6bcdfca3a3324c51a7a99e91e4f1260e"
format_stamp: "Formatted at 2026-08-09 02:02:26 on dist-test-slave-ss4n"
server_key: "f18ddf3df6032aa2fe87b6f299167c70"
server_key_iv: "0256ddfdb6be6d7480fdadd95ed8448d"
server_key_version: "encryptionkey@0"
I20260809 02:02:26.252005 2476 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786240941968412-1795-0/minicluster-data/ts-0/wal/instance:
uuid: "6bcdfca3a3324c51a7a99e91e4f1260e"
format_stamp: "Formatted at 2026-08-09 02:02:26 on dist-test-slave-ss4n"
server_key: "f18ddf3df6032aa2fe87b6f299167c70"
server_key_iv: "0256ddfdb6be6d7480fdadd95ed8448d"
server_key_version: "encryptionkey@0"
I20260809 02:02:26.256201 2476 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 02:02:26.259018 2491 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:26.260386 2476 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 02:02:26.260500 2476 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786240941968412-1795-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786240941968412-1795-0/minicluster-data/ts-0/wal
uuid: "6bcdfca3a3324c51a7a99e91e4f1260e"
format_stamp: "Formatted at 2026-08-09 02:02:26 on dist-test-slave-ss4n"
server_key: "f18ddf3df6032aa2fe87b6f299167c70"
server_key_iv: "0256ddfdb6be6d7480fdadd95ed8448d"
server_key_version: "encryptionkey@0"
I20260809 02:02:26.260632 2476 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786240941968412-1795-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786240941968412-1795-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786240941968412-1795-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:02:26.271334 2476 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:02:26.274495 2476 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:02:26.274662 2476 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:02:26.275197 2476 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:02:26.275990 2476 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:02:26.276036 2476 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:26.276077 2476 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:02:26.276100 2476 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:26.284960 2476 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.193:42921
I20260809 02:02:26.285034 2611 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.193:42921 every 8 connection(s)
I20260809 02:02:26.286013 2476 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786240941968412-1795-0/minicluster-data/ts-0/data/info.pb
I20260809 02:02:26.292800 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 2476
I20260809 02:02:26.292938 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786240941968412-1795-0/minicluster-data/ts-0/wal/instance
I20260809 02:02:26.293264 1795 external_mini_cluster.cc:1428] Setting key dba7f517dc290088d4ad9cd8b33c565a
Aug 09 02:02:26 dist-test-slave-ss4n krb5kdc[2363](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240946, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.193@KRBTEST.COM for kudu/127.1.192.254@KRBTEST.COM
I20260809 02:02:26.301815 2452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:26.288007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.193:55101 (local address 127.1.192.254:34951)
0809 02:02:26.288170 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:26.288176 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:26.289073 (+ 897us) server_negotiation.cc:407] Connection header received
0809 02:02:26.290726 (+ 1653us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:26.290729 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:26.290782 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:26.290861 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:26.292099 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:26.292662 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:26.294259 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:26.294587 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:26.294703 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:26.297853 (+ 3150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:02:26.297856 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:02:26.297858 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:02:26.297884 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:02:26.299431 (+ 1547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:26.299998 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:26.300007 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:26.300008 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:26.300086 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:26.300453 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:26.300456 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:26.300458 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:26.300762 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:02:26.300868 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:26.301411 (+ 543us) server_negotiation.cc:299] Negotiation successful
0809 02:02:26.301586 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 02:02:26.302718 2614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:26.288419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:34951 (local address 127.1.192.193:55101)
0809 02:02:26.288904 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:26.288949 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 02:02:26.290504 (+ 1555us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:02:26.291060 (+ 556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:02:26.291067 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:02:26.291463 (+ 396us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:02:26.291951 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:26.291964 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:26.292812 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:26.292819 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:26.294137 (+ 1318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:26.294147 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:26.294876 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:26.294880 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:26.295103 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:26.295779 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:02:26.295806 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:02:26.297659 (+ 1853us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:02:26.299566 (+ 1907us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:26.299573 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:26.299591 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:26.299877 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:26.300184 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:26.300188 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:26.300190 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:26.300347 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:26.300868 (+ 521us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:02:26.300877 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:02:26.301156 (+ 279us) client_negotiation.cc:769] Sending connection context
0809 02:02:26.301352 (+ 196us) client_negotiation.cc:240] Negotiation successful
0809 02:02:26.301767 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":127,"threads_started":1}
WARNING: no policy specified for kudu/127.1.192.194@KRBTEST.COM; defaulting to no policy
I20260809 02:02:26.304024 2612 heartbeater.cc:347] Connected to a master server at 127.1.192.254:34951
I20260809 02:02:26.304272 2612 heartbeater.cc:464] Registering TS with master...
I20260809 02:02:26.304793 2612 heartbeater.cc:511] Master 127.1.192.254:34951 requested a full tablet report, sending...
I20260809 02:02:26.306437 2411 ts_manager.cc:194] Registered new tserver with Master: 6bcdfca3a3324c51a7a99e91e4f1260e (127.1.192.193:42921)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.192.194@KRBTEST.COM" created.
I20260809 02:02:26.307713 2411 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.192.193@KRBTEST.COM'} at 127.1.192.193:55101
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.192.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.194.keytab.
Entry for principal kudu/127.1.192.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.194.keytab.
I20260809 02:02:26.326164 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786240941968412-1795-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786240941968412-1795-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786240941968412-1795-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786240941968412-1795-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.192.194:0
--local_ip_for_outbound_sockets=127.1.192.194
--webserver_interface=127.1.192.194
--webserver_port=0
--tserver_master_addrs=127.1.192.254:34951
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.194.keytab
--principal=kudu/127.1.192.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:41551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:02:26.441243 2617 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:02:26.441603 2617 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:02:26.441679 2617 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:02:26.445995 2617 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:02:26.446065 2617 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:02:26.446167 2617 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.192.194
I20260809 02:02:26.451805 2617 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:41551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786240941968412-1795-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786240941968412-1795-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.194.keytab
--principal=kudu/127.1.192.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.192.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786240941968412-1795-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.192.194
--webserver_port=0
--tserver_master_addrs=127.1.192.254:34951
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2617
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.192.194
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786240941968412-1795-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:02:26.453084 2617 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:02:26.454246 2617 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:02:26.464396 2623 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:02:26.464390 2625 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:02:26.464504 2622 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:02:26.464576 2617 server_base.cc:1034] running on GCE node
I20260809 02:02:26.465230 2617 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:02:26.465952 2617 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:02:26.467154 2617 hybrid_clock.cc:648] HybridClock initialized: now 1786240946467143 us; error 30 us; skew 500 ppm
Aug 09 02:02:26 dist-test-slave-ss4n krb5kdc[2363](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240946, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:02:26.470014 2617 init.cc:377] Logged in from keytab as kudu/127.1.192.194@KRBTEST.COM (short username kudu)
I20260809 02:02:26.470927 2617 webserver.cc:460] Webserver started at http://127.1.192.194:35611/ using document root <none> and password file <none>
I20260809 02:02:26.471508 2617 fs_manager.cc:359] Metadata directory not provided
I20260809 02:02:26.471602 2617 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:02:26.471839 2617 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:02:26.473994 2617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786240941968412-1795-0/minicluster-data/ts-1/data/instance:
uuid: "3795ad5dae7c4727abcbf91d8fa35580"
format_stamp: "Formatted at 2026-08-09 02:02:26 on dist-test-slave-ss4n"
server_key: "39f05df65346397bd59cdd5068b3097c"
server_key_iv: "416085bfef9d7d55bff88fd4dcede286"
server_key_version: "encryptionkey@0"
I20260809 02:02:26.474581 2617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786240941968412-1795-0/minicluster-data/ts-1/wal/instance:
uuid: "3795ad5dae7c4727abcbf91d8fa35580"
format_stamp: "Formatted at 2026-08-09 02:02:26 on dist-test-slave-ss4n"
server_key: "39f05df65346397bd59cdd5068b3097c"
server_key_iv: "416085bfef9d7d55bff88fd4dcede286"
server_key_version: "encryptionkey@0"
I20260809 02:02:26.478765 2617 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260809 02:02:26.481606 2632 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:26.482867 2617 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 02:02:26.483011 2617 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786240941968412-1795-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786240941968412-1795-0/minicluster-data/ts-1/wal
uuid: "3795ad5dae7c4727abcbf91d8fa35580"
format_stamp: "Formatted at 2026-08-09 02:02:26 on dist-test-slave-ss4n"
server_key: "39f05df65346397bd59cdd5068b3097c"
server_key_iv: "416085bfef9d7d55bff88fd4dcede286"
server_key_version: "encryptionkey@0"
I20260809 02:02:26.483153 2617 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786240941968412-1795-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786240941968412-1795-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786240941968412-1795-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:02:26.503549 2617 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:02:26.506773 2617 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:02:26.506997 2617 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:02:26.507632 2617 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:02:26.508490 2617 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:02:26.508564 2617 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:26.508630 2617 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:02:26.508679 2617 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:26.518141 2617 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.194:39707
I20260809 02:02:26.518163 2755 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.194:39707 every 8 connection(s)
I20260809 02:02:26.519076 2617 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786240941968412-1795-0/minicluster-data/ts-1/data/info.pb
I20260809 02:02:26.524391 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 2617
I20260809 02:02:26.524511 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786240941968412-1795-0/minicluster-data/ts-1/wal/instance
I20260809 02:02:26.524793 1795 external_mini_cluster.cc:1428] Setting key 13da77dc796c1351ffb6f77a42992356
Aug 09 02:02:26 dist-test-slave-ss4n krb5kdc[2363](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240946, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.194@KRBTEST.COM for kudu/127.1.192.254@KRBTEST.COM
I20260809 02:02:26.534366 2452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:26.521155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.194:51905 (local address 127.1.192.254:34951)
0809 02:02:26.521320 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:26.521325 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:26.522341 (+ 1016us) server_negotiation.cc:407] Connection header received
0809 02:02:26.523722 (+ 1381us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:26.523725 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:26.523776 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:26.523859 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:26.525461 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:26.526101 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:26.527315 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:26.527715 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:26.527838 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:26.530551 (+ 2713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:02:26.530559 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:02:26.530562 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:02:26.530586 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:02:26.532121 (+ 1535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:26.532731 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:26.532734 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:26.532737 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:26.532823 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:26.533195 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:26.533200 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:26.533201 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:26.533403 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:02:26.533544 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:26.534059 (+ 515us) server_negotiation.cc:299] Negotiation successful
0809 02:02:26.534192 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
WARNING: no policy specified for kudu/127.1.192.195@KRBTEST.COM; defaulting to no policy
I20260809 02:02:26.535282 2758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:26.521673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:34951 (local address 127.1.192.194:51905)
0809 02:02:26.522154 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:26.522211 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0809 02:02:26.523487 (+ 1276us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:02:26.524032 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:02:26.524039 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:02:26.524652 (+ 613us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:02:26.525313 (+ 661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:26.525331 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:26.526257 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:26.526260 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:26.527191 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:26.527200 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:26.527879 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:26.527881 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:26.528037 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:26.528766 (+ 729us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:02:26.528782 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:02:26.530358 (+ 1576us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:02:26.532271 (+ 1913us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:26.532277 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:26.532294 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:26.532568 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:26.532925 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:26.532929 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:26.532931 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:26.533082 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:26.533543 (+ 461us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:02:26.533550 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:02:26.533814 (+ 264us) client_negotiation.cc:769] Sending connection context
0809 02:02:26.534018 (+ 204us) client_negotiation.cc:240] Negotiation successful
0809 02:02:26.534384 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":123,"threads_started":1}
I20260809 02:02:26.536818 2756 heartbeater.cc:347] Connected to a master server at 127.1.192.254:34951
I20260809 02:02:26.537163 2756 heartbeater.cc:464] Registering TS with master...
I20260809 02:02:26.537959 2756 heartbeater.cc:511] Master 127.1.192.254:34951 requested a full tablet report, sending...
I20260809 02:02:26.539132 2411 ts_manager.cc:194] Registered new tserver with Master: 3795ad5dae7c4727abcbf91d8fa35580 (127.1.192.194:39707)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.192.195@KRBTEST.COM" created.
I20260809 02:02:26.539811 2411 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.192.194@KRBTEST.COM'} at 127.1.192.194:51905
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.192.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.195.keytab.
Entry for principal kudu/127.1.192.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.195.keytab.
I20260809 02:02:26.561002 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786240941968412-1795-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786240941968412-1795-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786240941968412-1795-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786240941968412-1795-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.192.195:0
--local_ip_for_outbound_sockets=127.1.192.195
--webserver_interface=127.1.192.195
--webserver_port=0
--tserver_master_addrs=127.1.192.254:34951
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.195.keytab
--principal=kudu/127.1.192.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:41551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:02:26.677347 2762 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:02:26.677785 2762 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:02:26.677876 2762 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:02:26.682087 2762 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:02:26.682205 2762 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:02:26.682303 2762 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.192.195
I20260809 02:02:26.689126 2762 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:41551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786240941968412-1795-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786240941968412-1795-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.195.keytab
--principal=kudu/127.1.192.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.192.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786240941968412-1795-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.192.195
--webserver_port=0
--tserver_master_addrs=127.1.192.254:34951
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2762
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.192.195
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786240941968412-1795-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:02:26.690546 2762 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:02:26.691998 2762 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:02:26.703296 2767 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:02:26.703296 2768 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:02:26.703475 2762 server_base.cc:1034] running on GCE node
W20260809 02:02:26.703320 2770 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:02:26.704104 2762 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:02:26.704922 2762 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:02:26.706168 2762 hybrid_clock.cc:648] HybridClock initialized: now 1786240946706124 us; error 62 us; skew 500 ppm
Aug 09 02:02:26 dist-test-slave-ss4n krb5kdc[2363](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240946, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:02:26.710410 2762 init.cc:377] Logged in from keytab as kudu/127.1.192.195@KRBTEST.COM (short username kudu)
I20260809 02:02:26.711419 2762 webserver.cc:460] Webserver started at http://127.1.192.195:39611/ using document root <none> and password file <none>
I20260809 02:02:26.712038 2762 fs_manager.cc:359] Metadata directory not provided
I20260809 02:02:26.712106 2762 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:02:26.712345 2762 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:02:26.714646 2762 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786240941968412-1795-0/minicluster-data/ts-2/data/instance:
uuid: "54cfe28c4b3242c2aeff4e7f2ce0d7f3"
format_stamp: "Formatted at 2026-08-09 02:02:26 on dist-test-slave-ss4n"
server_key: "eab237a0865b1169dd1498d6d8932a01"
server_key_iv: "629a3c66efb9ad418637eaebdbc7a382"
server_key_version: "encryptionkey@0"
I20260809 02:02:26.715265 2762 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786240941968412-1795-0/minicluster-data/ts-2/wal/instance:
uuid: "54cfe28c4b3242c2aeff4e7f2ce0d7f3"
format_stamp: "Formatted at 2026-08-09 02:02:26 on dist-test-slave-ss4n"
server_key: "eab237a0865b1169dd1498d6d8932a01"
server_key_iv: "629a3c66efb9ad418637eaebdbc7a382"
server_key_version: "encryptionkey@0"
I20260809 02:02:26.719403 2762 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260809 02:02:26.722056 2777 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:26.723644 2762 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 02:02:26.723809 2762 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786240941968412-1795-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786240941968412-1795-0/minicluster-data/ts-2/wal
uuid: "54cfe28c4b3242c2aeff4e7f2ce0d7f3"
format_stamp: "Formatted at 2026-08-09 02:02:26 on dist-test-slave-ss4n"
server_key: "eab237a0865b1169dd1498d6d8932a01"
server_key_iv: "629a3c66efb9ad418637eaebdbc7a382"
server_key_version: "encryptionkey@0"
I20260809 02:02:26.723954 2762 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786240941968412-1795-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786240941968412-1795-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786240941968412-1795-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:02:26.745321 2762 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:02:26.749141 2762 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:02:26.749415 2762 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:02:26.750217 2762 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:02:26.751220 2762 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:02:26.751329 2762 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:26.751406 2762 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:02:26.751482 2762 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:26.762552 2762 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.195:38821
I20260809 02:02:26.762583 2897 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.195:38821 every 8 connection(s)
I20260809 02:02:26.763635 2762 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786240941968412-1795-0/minicluster-data/ts-2/data/info.pb
I20260809 02:02:26.769865 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 2762
I20260809 02:02:26.770020 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786240941968412-1795-0/minicluster-data/ts-2/wal/instance
I20260809 02:02:26.770308 1795 external_mini_cluster.cc:1428] Setting key c0981d8aac713b43f73eb2fcf2b9002b
Aug 09 02:02:26 dist-test-slave-ss4n krb5kdc[2363](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240946, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.195@KRBTEST.COM for kudu/127.1.192.254@KRBTEST.COM
I20260809 02:02:26.778821 2452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:26.765708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.195:39073 (local address 127.1.192.254:34951)
0809 02:02:26.765888 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:26.765894 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:26.766932 (+ 1038us) server_negotiation.cc:407] Connection header received
0809 02:02:26.768303 (+ 1371us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:26.768307 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:26.768364 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:26.768454 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:26.769937 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:26.770682 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:26.771988 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:26.772241 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:26.772369 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:26.775172 (+ 2803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:02:26.775176 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:02:26.775178 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:02:26.775206 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:02:26.776658 (+ 1452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:26.777226 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:26.777228 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:26.777230 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:26.777312 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:26.777705 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:26.777707 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:26.777708 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:26.777892 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:02:26.777975 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:26.778548 (+ 573us) server_negotiation.cc:299] Negotiation successful
0809 02:02:26.778671 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 02:02:26.779691 2900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:26.766209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:34951 (local address 127.1.192.195:39073)
0809 02:02:26.766742 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:26.766789 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 02:02:26.768047 (+ 1258us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:02:26.768669 (+ 622us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:02:26.768676 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:02:26.769071 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:02:26.769724 (+ 653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:26.769747 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:26.770841 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:26.770845 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:26.771839 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:26.771849 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:26.772437 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:26.772442 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:26.772658 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:26.773271 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:02:26.773290 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:02:26.774944 (+ 1654us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:02:26.776782 (+ 1838us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:26.776787 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:26.776801 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:26.777083 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:26.777403 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:26.777405 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:26.777406 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:26.777609 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:26.777982 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:02:26.777988 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:02:26.778261 (+ 273us) client_negotiation.cc:769] Sending connection context
0809 02:02:26.778420 (+ 159us) client_negotiation.cc:240] Negotiation successful
0809 02:02:26.778782 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":138,"threads_started":1}
I20260809 02:02:26.780927 2898 heartbeater.cc:347] Connected to a master server at 127.1.192.254:34951
I20260809 02:02:26.781210 2898 heartbeater.cc:464] Registering TS with master...
I20260809 02:02:26.781874 2898 heartbeater.cc:511] Master 127.1.192.254:34951 requested a full tablet report, sending...
I20260809 02:02:26.783083 2411 ts_manager.cc:194] Registered new tserver with Master: 54cfe28c4b3242c2aeff4e7f2ce0d7f3 (127.1.192.195:38821)
I20260809 02:02:26.783905 2411 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.192.195@KRBTEST.COM'} at 127.1.192.195:39073
I20260809 02:02:26.784790 1795 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 02:02:26.798709 2452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:26.789831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37376 (local address 127.1.192.254:34951)
0809 02:02:26.789981 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:26.789985 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:26.790275 (+ 290us) server_negotiation.cc:407] Connection header received
0809 02:02:26.790374 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:26.790377 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:26.790433 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:26.790581 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:26.791412 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:26.792002 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:26.792987 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:26.793221 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:26.793306 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:26.794940 (+ 1634us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:02:26.794943 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:02:26.794945 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:02:26.794972 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:02:26.796466 (+ 1494us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:26.796954 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:26.796957 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:26.796958 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:26.797061 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:26.797564 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:26.797572 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:26.797575 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:26.797942 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:02:26.798084 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:26.798372 (+ 288us) server_negotiation.cc:299] Negotiation successful
0809 02:02:26.798564 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 02:02:26.805652 2411 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37376:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 02:02:26.809026 2411 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:02:26.826575 2909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:26.820006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.195:38821 (local address 127.0.0.1:55832)
0809 02:02:26.821072 (+ 1066us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:26.821086 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 02:02:26.821165 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:02:26.821987 (+ 822us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:02:26.821993 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:02:26.822016 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:02:26.822191 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:26.822199 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:26.823413 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:26.823417 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:26.824783 (+ 1366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:26.824794 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:26.825895 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:26.825899 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:26.826030 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:26.826074 (+ 44us) client_negotiation.cc:769] Sending connection context
0809 02:02:26.826236 (+ 162us) client_negotiation.cc:240] Negotiation successful
0809 02:02:26.826384 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1005,"mutex_wait_us":1,"thread_start_us":92,"threads_started":1}
I20260809 02:02:26.826620 2914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:26.820804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55832 (local address 127.1.192.195:38821)
0809 02:02:26.821462 (+ 658us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:26.821499 (+ 37us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:26.821528 (+ 29us) server_negotiation.cc:407] Connection header received
0809 02:02:26.821629 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:26.821635 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:26.821831 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:26.821997 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:02:26.822306 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:26.823253 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:26.824932 (+ 1679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:26.825752 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:26.825875 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:26.826129 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:26.826287 (+ 158us) server_negotiation.cc:299] Negotiation successful
0809 02:02:26.826405 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":553,"thread_start_us":107,"threads_started":1}
I20260809 02:02:26.827430 2910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:26.820312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.194:39707 (local address 127.0.0.1:38852)
0809 02:02:26.821462 (+ 1150us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:26.821499 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 02:02:26.821628 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:02:26.822256 (+ 628us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:02:26.822262 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:02:26.822286 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:02:26.822391 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:26.822398 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:26.823857 (+ 1459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:26.823860 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:26.825288 (+ 1428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:26.825300 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:26.826979 (+ 1679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:26.826984 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:26.827082 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:26.827115 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 02:02:26.827187 (+ 72us) client_negotiation.cc:240] Negotiation successful
0809 02:02:26.827276 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1095,"mutex_wait_us":173,"thread_start_us":100,"threads_started":1}
I20260809 02:02:26.827723 2911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:26.820090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43194 (local address 127.1.192.193:42921)
0809 02:02:26.821441 (+ 1351us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:26.821448 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:26.821473 (+ 25us) server_negotiation.cc:407] Connection header received
0809 02:02:26.821662 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:26.821670 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:26.821896 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:26.822059 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:02:26.822471 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:26.823702 (+ 1231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:26.825456 (+ 1754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:26.826391 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:26.826512 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:26.827092 (+ 580us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:26.827457 (+ 365us) server_negotiation.cc:299] Negotiation successful
0809 02:02:26.827578 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1214,"thread_start_us":113,"threads_started":1}
I20260809 02:02:26.827883 2913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:26.820645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38852 (local address 127.1.192.194:39707)
0809 02:02:26.821254 (+ 609us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:26.821268 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:26.821615 (+ 347us) server_negotiation.cc:407] Connection header received
0809 02:02:26.821731 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:26.821736 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:26.821895 (+ 159us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:26.822059 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:02:26.822521 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:26.823702 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:26.825589 (+ 1887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:26.826396 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:26.826499 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:26.826644 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:26.827608 (+ 964us) server_negotiation.cc:299] Negotiation successful
0809 02:02:26.827727 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":494,"thread_start_us":160,"threads_started":1}
I20260809 02:02:26.828150 2912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:26.819937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.193:42921 (local address 127.0.0.1:43194)
0809 02:02:26.820888 (+ 951us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:26.820947 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0809 02:02:26.821095 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:02:26.822193 (+ 1098us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:02:26.822195 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:02:26.822212 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:02:26.822322 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:26.822329 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:26.823857 (+ 1528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:26.823860 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:26.825290 (+ 1430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:26.825300 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:26.827168 (+ 1868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:26.827172 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:26.827285 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:26.827311 (+ 26us) client_negotiation.cc:769] Sending connection context
0809 02:02:26.827402 (+ 91us) client_negotiation.cc:240] Negotiation successful
0809 02:02:26.827692 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":746,"mutex_wait_us":114,"thread_start_us":97,"threads_started":1}
I20260809 02:02:26.830904 2543 tablet_service.cc:1467] Processing CreateTablet for tablet e3a7b7e4470543229e9f58d0bbc231e6 (DEFAULT_TABLE table=test-table [id=2c4db94757ee4f26bd0bc20d1ea0fcd9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:02:26.830991 2686 tablet_service.cc:1467] Processing CreateTablet for tablet e3a7b7e4470543229e9f58d0bbc231e6 (DEFAULT_TABLE table=test-table [id=2c4db94757ee4f26bd0bc20d1ea0fcd9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:02:26.830981 2830 tablet_service.cc:1467] Processing CreateTablet for tablet e3a7b7e4470543229e9f58d0bbc231e6 (DEFAULT_TABLE table=test-table [id=2c4db94757ee4f26bd0bc20d1ea0fcd9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:02:26.832162 2543 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3a7b7e4470543229e9f58d0bbc231e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:02:26.832217 2830 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3a7b7e4470543229e9f58d0bbc231e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:02:26.832217 2686 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3a7b7e4470543229e9f58d0bbc231e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:02:26.838899 2915 tablet_bootstrap.cc:492] T e3a7b7e4470543229e9f58d0bbc231e6 P 6bcdfca3a3324c51a7a99e91e4f1260e: Bootstrap starting.
I20260809 02:02:26.840497 2918 tablet_bootstrap.cc:492] T e3a7b7e4470543229e9f58d0bbc231e6 P 54cfe28c4b3242c2aeff4e7f2ce0d7f3: Bootstrap starting.
I20260809 02:02:26.840487 2916 tablet_bootstrap.cc:492] T e3a7b7e4470543229e9f58d0bbc231e6 P 3795ad5dae7c4727abcbf91d8fa35580: Bootstrap starting.
I20260809 02:02:26.841754 2915 tablet_bootstrap.cc:654] T e3a7b7e4470543229e9f58d0bbc231e6 P 6bcdfca3a3324c51a7a99e91e4f1260e: Neither blocks nor log segments found. Creating new log.
I20260809 02:02:26.843492 2915 log.cc:824] T e3a7b7e4470543229e9f58d0bbc231e6 P 6bcdfca3a3324c51a7a99e91e4f1260e: Log is configured to *not* fsync() on all Append() calls
I20260809 02:02:26.843942 2918 tablet_bootstrap.cc:654] T e3a7b7e4470543229e9f58d0bbc231e6 P 54cfe28c4b3242c2aeff4e7f2ce0d7f3: Neither blocks nor log segments found. Creating new log.
I20260809 02:02:26.844501 2916 tablet_bootstrap.cc:654] T e3a7b7e4470543229e9f58d0bbc231e6 P 3795ad5dae7c4727abcbf91d8fa35580: Neither blocks nor log segments found. Creating new log.
I20260809 02:02:26.845541 2918 log.cc:824] T e3a7b7e4470543229e9f58d0bbc231e6 P 54cfe28c4b3242c2aeff4e7f2ce0d7f3: Log is configured to *not* fsync() on all Append() calls
I20260809 02:02:26.845546 2916 log.cc:824] T e3a7b7e4470543229e9f58d0bbc231e6 P 3795ad5dae7c4727abcbf91d8fa35580: Log is configured to *not* fsync() on all Append() calls
I20260809 02:02:26.846021 2915 tablet_bootstrap.cc:492] T e3a7b7e4470543229e9f58d0bbc231e6 P 6bcdfca3a3324c51a7a99e91e4f1260e: No bootstrap required, opened a new log
I20260809 02:02:26.846220 2915 ts_tablet_manager.cc:1397] T e3a7b7e4470543229e9f58d0bbc231e6 P 6bcdfca3a3324c51a7a99e91e4f1260e: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260809 02:02:26.848014 2916 tablet_bootstrap.cc:492] T e3a7b7e4470543229e9f58d0bbc231e6 P 3795ad5dae7c4727abcbf91d8fa35580: No bootstrap required, opened a new log
I20260809 02:02:26.848038 2918 tablet_bootstrap.cc:492] T e3a7b7e4470543229e9f58d0bbc231e6 P 54cfe28c4b3242c2aeff4e7f2ce0d7f3: No bootstrap required, opened a new log
I20260809 02:02:26.848240 2918 ts_tablet_manager.cc:1397] T e3a7b7e4470543229e9f58d0bbc231e6 P 54cfe28c4b3242c2aeff4e7f2ce0d7f3: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260809 02:02:26.848240 2916 ts_tablet_manager.cc:1397] T e3a7b7e4470543229e9f58d0bbc231e6 P 3795ad5dae7c4727abcbf91d8fa35580: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260809 02:02:26.852972 2916 raft_consensus.cc:348] T e3a7b7e4470543229e9f58d0bbc231e6 P 3795ad5dae7c4727abcbf91d8fa35580 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bcdfca3a3324c51a7a99e91e4f1260e" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 42921 } } peers { permanent_uuid: "54cfe28c4b3242c2aeff4e7f2ce0d7f3" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 38821 } } peers { permanent_uuid: "3795ad5dae7c4727abcbf91d8fa35580" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 39707 } }
I20260809 02:02:26.852972 2918 raft_consensus.cc:348] T e3a7b7e4470543229e9f58d0bbc231e6 P 54cfe28c4b3242c2aeff4e7f2ce0d7f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bcdfca3a3324c51a7a99e91e4f1260e" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 42921 } } peers { permanent_uuid: "54cfe28c4b3242c2aeff4e7f2ce0d7f3" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 38821 } } peers { permanent_uuid: "3795ad5dae7c4727abcbf91d8fa35580" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 39707 } }
I20260809 02:02:26.853351 2916 raft_consensus.cc:374] T e3a7b7e4470543229e9f58d0bbc231e6 P 3795ad5dae7c4727abcbf91d8fa35580 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:02:26.853351 2918 raft_consensus.cc:374] T e3a7b7e4470543229e9f58d0bbc231e6 P 54cfe28c4b3242c2aeff4e7f2ce0d7f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:02:26.853206 2915 raft_consensus.cc:348] T e3a7b7e4470543229e9f58d0bbc231e6 P 6bcdfca3a3324c51a7a99e91e4f1260e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bcdfca3a3324c51a7a99e91e4f1260e" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 42921 } } peers { permanent_uuid: "54cfe28c4b3242c2aeff4e7f2ce0d7f3" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 38821 } } peers { permanent_uuid: "3795ad5dae7c4727abcbf91d8fa35580" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 39707 } }
I20260809 02:02:26.853536 2915 raft_consensus.cc:374] T e3a7b7e4470543229e9f58d0bbc231e6 P 6bcdfca3a3324c51a7a99e91e4f1260e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:02:26.853440 2916 raft_consensus.cc:729] T e3a7b7e4470543229e9f58d0bbc231e6 P 3795ad5dae7c4727abcbf91d8fa35580 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3795ad5dae7c4727abcbf91d8fa35580, State: Initialized, Role: FOLLOWER
I20260809 02:02:26.853436 2918 raft_consensus.cc:729] T e3a7b7e4470543229e9f58d0bbc231e6 P 54cfe28c4b3242c2aeff4e7f2ce0d7f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54cfe28c4b3242c2aeff4e7f2ce0d7f3, State: Initialized, Role: FOLLOWER
I20260809 02:02:26.853617 2915 raft_consensus.cc:729] T e3a7b7e4470543229e9f58d0bbc231e6 P 6bcdfca3a3324c51a7a99e91e4f1260e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bcdfca3a3324c51a7a99e91e4f1260e, State: Initialized, Role: FOLLOWER
I20260809 02:02:26.854240 2918 consensus_queue.cc:260] T e3a7b7e4470543229e9f58d0bbc231e6 P 54cfe28c4b3242c2aeff4e7f2ce0d7f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bcdfca3a3324c51a7a99e91e4f1260e" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 42921 } } peers { permanent_uuid: "54cfe28c4b3242c2aeff4e7f2ce0d7f3" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 38821 } } peers { permanent_uuid: "3795ad5dae7c4727abcbf91d8fa35580" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 39707 } }
I20260809 02:02:26.854260 2915 consensus_queue.cc:260] T e3a7b7e4470543229e9f58d0bbc231e6 P 6bcdfca3a3324c51a7a99e91e4f1260e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bcdfca3a3324c51a7a99e91e4f1260e" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 42921 } } peers { permanent_uuid: "54cfe28c4b3242c2aeff4e7f2ce0d7f3" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 38821 } } peers { permanent_uuid: "3795ad5dae7c4727abcbf91d8fa35580" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 39707 } }
I20260809 02:02:26.854460 2916 consensus_queue.cc:260] T e3a7b7e4470543229e9f58d0bbc231e6 P 3795ad5dae7c4727abcbf91d8fa35580 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bcdfca3a3324c51a7a99e91e4f1260e" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 42921 } } peers { permanent_uuid: "54cfe28c4b3242c2aeff4e7f2ce0d7f3" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 38821 } } peers { permanent_uuid: "3795ad5dae7c4727abcbf91d8fa35580" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 39707 } }
I20260809 02:02:26.855197 2612 heartbeater.cc:503] Master 127.1.192.254:34951 was elected leader, sending a full tablet report...
I20260809 02:02:26.855583 2915 ts_tablet_manager.cc:1428] T e3a7b7e4470543229e9f58d0bbc231e6 P 6bcdfca3a3324c51a7a99e91e4f1260e: Time spent starting tablet: real 0.009s user 0.005s sys 0.004s
I20260809 02:02:26.855599 2916 ts_tablet_manager.cc:1428] T e3a7b7e4470543229e9f58d0bbc231e6 P 3795ad5dae7c4727abcbf91d8fa35580: Time spent starting tablet: real 0.007s user 0.005s sys 0.003s
I20260809 02:02:26.855599 2918 ts_tablet_manager.cc:1428] T e3a7b7e4470543229e9f58d0bbc231e6 P 54cfe28c4b3242c2aeff4e7f2ce0d7f3: Time spent starting tablet: real 0.007s user 0.005s sys 0.003s
I20260809 02:02:26.856006 2756 heartbeater.cc:503] Master 127.1.192.254:34951 was elected leader, sending a full tablet report...
I20260809 02:02:26.856006 2898 heartbeater.cc:503] Master 127.1.192.254:34951 was elected leader, sending a full tablet report...
W20260809 02:02:27.015096 2899 tablet.cc:2369] T e3a7b7e4470543229e9f58d0bbc231e6 P 54cfe28c4b3242c2aeff4e7f2ce0d7f3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 02:02:27.020296 2757 tablet.cc:2369] T e3a7b7e4470543229e9f58d0bbc231e6 P 3795ad5dae7c4727abcbf91d8fa35580: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 02:02:27.037539 2613 tablet.cc:2369] T e3a7b7e4470543229e9f58d0bbc231e6 P 6bcdfca3a3324c51a7a99e91e4f1260e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 02:02:27.046998 2923 raft_consensus.cc:482] T e3a7b7e4470543229e9f58d0bbc231e6 P 6bcdfca3a3324c51a7a99e91e4f1260e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:02:27.047219 2923 raft_consensus.cc:504] T e3a7b7e4470543229e9f58d0bbc231e6 P 6bcdfca3a3324c51a7a99e91e4f1260e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bcdfca3a3324c51a7a99e91e4f1260e" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 42921 } } peers { permanent_uuid: "54cfe28c4b3242c2aeff4e7f2ce0d7f3" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 38821 } } peers { permanent_uuid: "3795ad5dae7c4727abcbf91d8fa35580" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 39707 } }
I20260809 02:02:27.048496 2923 leader_election.cc:288] T e3a7b7e4470543229e9f58d0bbc231e6 P 6bcdfca3a3324c51a7a99e91e4f1260e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 54cfe28c4b3242c2aeff4e7f2ce0d7f3 (127.1.192.195:38821), 3795ad5dae7c4727abcbf91d8fa35580 (127.1.192.194:39707)
I20260809 02:02:27.052841 2914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:27.048903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.193:40929 (local address 127.1.192.195:38821)
0809 02:02:27.049104 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:27.049109 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:27.049338 (+ 229us) server_negotiation.cc:407] Connection header received
0809 02:02:27.049543 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:27.049548 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:27.049610 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:27.049669 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:02:27.050064 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:27.050665 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:27.051814 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:27.052208 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:27.052340 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:27.052394 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:27.052614 (+ 220us) server_negotiation.cc:299] Negotiation successful
0809 02:02:27.052703 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260809 02:02:27.052893 2927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:27.048912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.194:39707 (local address 127.1.192.193:51425)
0809 02:02:27.049262 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:27.049274 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:02:27.049418 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:02:27.049776 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:02:27.049782 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:02:27.049825 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:02:27.049933 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:27.049938 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:27.050742 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:27.050747 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:27.051633 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:27.051643 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:27.052382 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:27.052385 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:27.052545 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:27.052591 (+ 46us) client_negotiation.cc:769] Sending connection context
0809 02:02:27.052672 (+ 81us) client_negotiation.cc:240] Negotiation successful
0809 02:02:27.052736 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"mutex_wait_us":47,"spinlock_wait_cycles":896,"thread_start_us":126,"threads_started":1}
I20260809 02:02:27.052937 2913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:27.049036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.193:51425 (local address 127.1.192.194:39707)
0809 02:02:27.049188 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:27.049193 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:27.049409 (+ 216us) server_negotiation.cc:407] Connection header received
0809 02:02:27.049586 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:27.049589 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:27.049640 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:27.049751 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:02:27.050057 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:27.050623 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:27.051779 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:27.052222 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:27.052367 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:27.052424 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:27.052720 (+ 296us) server_negotiation.cc:299] Negotiation successful
0809 02:02:27.052804 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 02:02:27.052891 2926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:27.048804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.195:38821 (local address 127.1.192.193:40929)
0809 02:02:27.049213 (+ 409us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:27.049229 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 02:02:27.049350 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:02:27.049711 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:02:27.049717 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:02:27.049740 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:02:27.049932 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:27.049939 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:27.050820 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:27.050823 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:27.051666 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:27.051674 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:27.052339 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:27.052343 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:27.052452 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:27.052501 (+ 49us) client_negotiation.cc:769] Sending connection context
0809 02:02:27.052576 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 02:02:27.052673 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"mutex_wait_us":1,"spinlock_wait_cycles":28544,"thread_start_us":125,"threads_started":1}
I20260809 02:02:27.053815 2706 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3a7b7e4470543229e9f58d0bbc231e6" candidate_uuid: "6bcdfca3a3324c51a7a99e91e4f1260e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3795ad5dae7c4727abcbf91d8fa35580" is_pre_election: true
I20260809 02:02:27.054219 2706 raft_consensus.cc:2454] T e3a7b7e4470543229e9f58d0bbc231e6 P 3795ad5dae7c4727abcbf91d8fa35580 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6bcdfca3a3324c51a7a99e91e4f1260e in term 0.
I20260809 02:02:27.054785 2494 leader_election.cc:302] T e3a7b7e4470543229e9f58d0bbc231e6 P 6bcdfca3a3324c51a7a99e91e4f1260e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3795ad5dae7c4727abcbf91d8fa35580, 6bcdfca3a3324c51a7a99e91e4f1260e; no voters:
I20260809 02:02:27.055054 2923 raft_consensus.cc:2781] T e3a7b7e4470543229e9f58d0bbc231e6 P 6bcdfca3a3324c51a7a99e91e4f1260e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:02:27.055186 2923 raft_consensus.cc:482] T e3a7b7e4470543229e9f58d0bbc231e6 P 6bcdfca3a3324c51a7a99e91e4f1260e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:02:27.055253 2923 raft_consensus.cc:3037] T e3a7b7e4470543229e9f58d0bbc231e6 P 6bcdfca3a3324c51a7a99e91e4f1260e [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:02:27.055778 2850 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3a7b7e4470543229e9f58d0bbc231e6" candidate_uuid: "6bcdfca3a3324c51a7a99e91e4f1260e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "54cfe28c4b3242c2aeff4e7f2ce0d7f3" is_pre_election: true
I20260809 02:02:27.056103 2850 raft_consensus.cc:2454] T e3a7b7e4470543229e9f58d0bbc231e6 P 54cfe28c4b3242c2aeff4e7f2ce0d7f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6bcdfca3a3324c51a7a99e91e4f1260e in term 0.
I20260809 02:02:27.056686 2923 raft_consensus.cc:504] T e3a7b7e4470543229e9f58d0bbc231e6 P 6bcdfca3a3324c51a7a99e91e4f1260e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bcdfca3a3324c51a7a99e91e4f1260e" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 42921 } } peers { permanent_uuid: "54cfe28c4b3242c2aeff4e7f2ce0d7f3" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 38821 } } peers { permanent_uuid: "3795ad5dae7c4727abcbf91d8fa35580" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 39707 } }
I20260809 02:02:27.057055 2923 leader_election.cc:288] T e3a7b7e4470543229e9f58d0bbc231e6 P 6bcdfca3a3324c51a7a99e91e4f1260e [CANDIDATE]: Term 1 election: Requested vote from peers 54cfe28c4b3242c2aeff4e7f2ce0d7f3 (127.1.192.195:38821), 3795ad5dae7c4727abcbf91d8fa35580 (127.1.192.194:39707)
I20260809 02:02:27.057507 2850 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3a7b7e4470543229e9f58d0bbc231e6" candidate_uuid: "6bcdfca3a3324c51a7a99e91e4f1260e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "54cfe28c4b3242c2aeff4e7f2ce0d7f3"
I20260809 02:02:27.057525 2706 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3a7b7e4470543229e9f58d0bbc231e6" candidate_uuid: "6bcdfca3a3324c51a7a99e91e4f1260e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3795ad5dae7c4727abcbf91d8fa35580"
I20260809 02:02:27.057658 2706 raft_consensus.cc:3037] T e3a7b7e4470543229e9f58d0bbc231e6 P 3795ad5dae7c4727abcbf91d8fa35580 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:02:27.057709 2850 raft_consensus.cc:3037] T e3a7b7e4470543229e9f58d0bbc231e6 P 54cfe28c4b3242c2aeff4e7f2ce0d7f3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:02:27.059155 2850 raft_consensus.cc:2454] T e3a7b7e4470543229e9f58d0bbc231e6 P 54cfe28c4b3242c2aeff4e7f2ce0d7f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6bcdfca3a3324c51a7a99e91e4f1260e in term 1.
I20260809 02:02:27.059161 2706 raft_consensus.cc:2454] T e3a7b7e4470543229e9f58d0bbc231e6 P 3795ad5dae7c4727abcbf91d8fa35580 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6bcdfca3a3324c51a7a99e91e4f1260e in term 1.
I20260809 02:02:27.059512 2494 leader_election.cc:302] T e3a7b7e4470543229e9f58d0bbc231e6 P 6bcdfca3a3324c51a7a99e91e4f1260e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 3795ad5dae7c4727abcbf91d8fa35580, 54cfe28c4b3242c2aeff4e7f2ce0d7f3, 6bcdfca3a3324c51a7a99e91e4f1260e; no voters:
I20260809 02:02:27.059762 2923 raft_consensus.cc:2781] T e3a7b7e4470543229e9f58d0bbc231e6 P 6bcdfca3a3324c51a7a99e91e4f1260e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:02:27.059993 2923 raft_consensus.cc:686] T e3a7b7e4470543229e9f58d0bbc231e6 P 6bcdfca3a3324c51a7a99e91e4f1260e [term 1 LEADER]: Becoming Leader. State: Replica: 6bcdfca3a3324c51a7a99e91e4f1260e, State: Running, Role: LEADER
I20260809 02:02:27.060369 2923 consensus_queue.cc:237] T e3a7b7e4470543229e9f58d0bbc231e6 P 6bcdfca3a3324c51a7a99e91e4f1260e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bcdfca3a3324c51a7a99e91e4f1260e" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 42921 } } peers { permanent_uuid: "54cfe28c4b3242c2aeff4e7f2ce0d7f3" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 38821 } } peers { permanent_uuid: "3795ad5dae7c4727abcbf91d8fa35580" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 39707 } }
I20260809 02:02:27.064184 2407 catalog_manager.cc:5495] T e3a7b7e4470543229e9f58d0bbc231e6 P 6bcdfca3a3324c51a7a99e91e4f1260e reported cstate change: term changed from 0 to 1, leader changed from <none> to 6bcdfca3a3324c51a7a99e91e4f1260e (127.1.192.193). New cstate: current_term: 1 leader_uuid: "6bcdfca3a3324c51a7a99e91e4f1260e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bcdfca3a3324c51a7a99e91e4f1260e" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 42921 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "54cfe28c4b3242c2aeff4e7f2ce0d7f3" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 38821 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3795ad5dae7c4727abcbf91d8fa35580" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 39707 } health_report { overall_health: UNKNOWN } } }
Aug 09 02:02:27 dist-test-slave-ss4n krb5kdc[2363](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240945, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.192.193@KRBTEST.COM
I20260809 02:02:27.131533 2911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:27.121582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43210 (local address 127.1.192.193:42921)
0809 02:02:27.121764 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:27.121770 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:27.121954 (+ 184us) server_negotiation.cc:407] Connection header received
0809 02:02:27.122135 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:27.122139 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:27.122200 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:27.122305 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:27.123048 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:27.123599 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:27.124482 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:27.124707 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:27.124828 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:27.127532 (+ 2704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:02:27.127567 (+ 35us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:02:27.127584 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:02:27.127611 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:02:27.129633 (+ 2022us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:27.130101 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:27.130113 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:27.130114 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:27.130226 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:27.130527 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:27.130531 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:27.130533 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:27.130809 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:02:27.131052 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:27.131225 (+ 173us) server_negotiation.cc:299] Negotiation successful
0809 02:02:27.131357 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
Aug 09 02:02:27 dist-test-slave-ss4n krb5kdc[2363](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240947, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 02:02:27 dist-test-slave-ss4n krb5kdc[2363](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240947, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.192.193@KRBTEST.COM
I20260809 02:02:27.161863 2911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:27.151907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43220 (local address 127.1.192.193:42921)
0809 02:02:27.152142 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:27.152147 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:27.152277 (+ 130us) server_negotiation.cc:407] Connection header received
0809 02:02:27.152471 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:27.152474 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:27.152525 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:27.152624 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:27.153355 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:27.154067 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:27.154957 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:27.155272 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:27.155396 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:27.158214 (+ 2818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:02:27.158217 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:02:27.158220 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:02:27.158247 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:02:27.159939 (+ 1692us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:27.160423 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:27.160426 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:27.160427 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:27.160511 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:27.160836 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:27.160838 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:27.160839 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:27.161027 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:02:27.161177 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:27.161596 (+ 419us) server_negotiation.cc:299] Negotiation successful
0809 02:02:27.161722 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111}
Aug 09 02:02:27 dist-test-slave-ss4n krb5kdc[2363](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240947, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 02:02:27 dist-test-slave-ss4n krb5kdc[2363](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240947, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.192.193@KRBTEST.COM
I20260809 02:02:27.191824 2911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:27.182819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43234 (local address 127.1.192.193:42921)
0809 02:02:27.182960 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:27.182965 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:27.183153 (+ 188us) server_negotiation.cc:407] Connection header received
0809 02:02:27.183340 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:27.183343 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:27.183388 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:27.183476 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:27.184190 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:27.184695 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:27.185593 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:27.185866 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:27.185975 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:27.188694 (+ 2719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:02:27.188697 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:02:27.188699 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:02:27.188723 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:02:27.190128 (+ 1405us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:27.190568 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:27.190578 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:27.190580 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:27.190668 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:27.190953 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:27.190958 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:27.190959 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:27.191137 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:02:27.191299 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:27.191524 (+ 225us) server_negotiation.cc:299] Negotiation successful
0809 02:02:27.191655 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
W20260809 02:02:27.193472 2522 tablet_service.cc:2908] Rejecting scan request for tablet e3a7b7e4470543229e9f58d0bbc231e6: Uninitialized: clean time has not yet been initialized
I20260809 02:02:27.195941 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 2476
I20260809 02:02:27.203616 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 2617
I20260809 02:02:27.210800 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 2762
I20260809 02:02:27.217736 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 2377
2026-08-09T02:02:27Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2663 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786240941968412-1795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:02:27 dist-test-slave-ss4n krb5kdc[2943](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:02:27 dist-test-slave-ss4n krb5kdc[2943](info): set up 2 sockets
Aug 09 02:02:27 dist-test-slave-ss4n krb5kdc[2943](info): commencing operation
krb5kdc: starting...
W20260809 02:02:28.335242 1795 mini_kdc.cc:121] Time spent starting KDC: real 1.092s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786240941968412-1795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786240941968412-1795-0/krb5kdc/test-user.keytab.
Aug 09 02:02:28 dist-test-slave-ss4n krb5kdc[2943](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240948, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:02:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:02:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.192.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.192.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.192.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab.
Entry for principal kudu/127.1.192.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab.
I20260809 02:02:28.469719 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786240941968412-1795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786240941968412-1795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786240941968412-1795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.192.254:39235
--webserver_interface=127.1.192.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab
--principal=kudu/127.1.192.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:43057
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.192.254:39235
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:02:28.584651 2958 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:02:28.585002 2958 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:02:28.585080 2958 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:02:28.589175 2958 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:02:28.589296 2958 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 02:02:28.589329 2958 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:02:28.589371 2958 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:02:28.589424 2958 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:02:28.589463 2958 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:02:28.595140 2958 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:43057
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786240941968412-1795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786240941968412-1795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.192.254:39235
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab
--principal=kudu/127.1.192.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.192.254:39235
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.192.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2958
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786240941968412-1795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:02:28.596354 2958 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:02:28.597399 2958 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:02:28.606334 2964 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:02:28.606334 2963 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:02:28.606563 2958 server_base.cc:1034] running on GCE node
W20260809 02:02:28.606392 2966 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:02:28.607383 2958 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:02:28.608559 2958 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:02:28.609762 2958 hybrid_clock.cc:648] HybridClock initialized: now 1786240948609733 us; error 35 us; skew 500 ppm
Aug 09 02:02:28 dist-test-slave-ss4n krb5kdc[2943](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240948, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:02:28.612860 2958 init.cc:377] Logged in from keytab as kudu/127.1.192.254@KRBTEST.COM (short username kudu)
I20260809 02:02:28.613898 2958 webserver.cc:460] Webserver started at http://127.1.192.254:40419/ using document root <none> and password file <none>
I20260809 02:02:28.614483 2958 fs_manager.cc:359] Metadata directory not provided
I20260809 02:02:28.614590 2958 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:02:28.614832 2958 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:02:28.616931 2958 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786240941968412-1795-0/minicluster-data/master-0/data/instance:
uuid: "484e26ab9179442793b9906de0294663"
format_stamp: "Formatted at 2026-08-09 02:02:28 on dist-test-slave-ss4n"
server_key: "d4c52c3091b3cdd5cb3e5d54ba627d8e"
server_key_iv: "1beac80a3b8acaac4bd73d6280dae215"
server_key_version: "encryptionkey@0"
I20260809 02:02:28.617631 2958 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786240941968412-1795-0/minicluster-data/master-0/wal/instance:
uuid: "484e26ab9179442793b9906de0294663"
format_stamp: "Formatted at 2026-08-09 02:02:28 on dist-test-slave-ss4n"
server_key: "d4c52c3091b3cdd5cb3e5d54ba627d8e"
server_key_iv: "1beac80a3b8acaac4bd73d6280dae215"
server_key_version: "encryptionkey@0"
I20260809 02:02:28.621554 2958 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 02:02:28.624861 2974 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:28.626359 2958 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 02:02:28.626523 2958 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786240941968412-1795-0/minicluster-data/master-0/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786240941968412-1795-0/minicluster-data/master-0/wal
uuid: "484e26ab9179442793b9906de0294663"
format_stamp: "Formatted at 2026-08-09 02:02:28 on dist-test-slave-ss4n"
server_key: "d4c52c3091b3cdd5cb3e5d54ba627d8e"
server_key_iv: "1beac80a3b8acaac4bd73d6280dae215"
server_key_version: "encryptionkey@0"
I20260809 02:02:28.626716 2958 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786240941968412-1795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786240941968412-1795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786240941968412-1795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:02:28.656117 2958 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:02:28.659864 2958 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:02:28.660100 2958 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:02:28.669961 2958 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.254:39235
I20260809 02:02:28.669968 3040 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.254:39235 every 8 connection(s)
I20260809 02:02:28.671129 2958 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
I20260809 02:02:28.674188 3041 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:02:28.678025 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 2958
I20260809 02:02:28.678158 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786240941968412-1795-0/minicluster-data/master-0/wal/instance
I20260809 02:02:28.678432 1795 external_mini_cluster.cc:1428] Setting key feef061abb99e7ffe114777e904857a4
I20260809 02:02:28.680357 3041 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 484e26ab9179442793b9906de0294663: Bootstrap starting.
I20260809 02:02:28.682993 3041 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 484e26ab9179442793b9906de0294663: Neither blocks nor log segments found. Creating new log.
I20260809 02:02:28.683823 3041 log.cc:824] T 00000000000000000000000000000000 P 484e26ab9179442793b9906de0294663: Log is configured to *not* fsync() on all Append() calls
I20260809 02:02:28.685851 3041 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 484e26ab9179442793b9906de0294663: No bootstrap required, opened a new log
Aug 09 02:02:28 dist-test-slave-ss4n krb5kdc[2943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240948, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.192.254@KRBTEST.COM
I20260809 02:02:28.689618 3041 raft_consensus.cc:348] T 00000000000000000000000000000000 P 484e26ab9179442793b9906de0294663 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "484e26ab9179442793b9906de0294663" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 39235 } }
I20260809 02:02:28.689957 3041 raft_consensus.cc:374] T 00000000000000000000000000000000 P 484e26ab9179442793b9906de0294663 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:02:28.690029 3041 raft_consensus.cc:729] T 00000000000000000000000000000000 P 484e26ab9179442793b9906de0294663 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 484e26ab9179442793b9906de0294663, State: Initialized, Role: FOLLOWER
I20260809 02:02:28.690572 3041 consensus_queue.cc:260] T 00000000000000000000000000000000 P 484e26ab9179442793b9906de0294663 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "484e26ab9179442793b9906de0294663" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 39235 } }
I20260809 02:02:28.690748 3041 raft_consensus.cc:388] T 00000000000000000000000000000000 P 484e26ab9179442793b9906de0294663 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:02:28.690819 3041 raft_consensus.cc:482] T 00000000000000000000000000000000 P 484e26ab9179442793b9906de0294663 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:02:28.690913 3041 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 484e26ab9179442793b9906de0294663 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:02:28.692071 3041 raft_consensus.cc:504] T 00000000000000000000000000000000 P 484e26ab9179442793b9906de0294663 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "484e26ab9179442793b9906de0294663" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 39235 } }
I20260809 02:02:28.692476 3041 leader_election.cc:302] T 00000000000000000000000000000000 P 484e26ab9179442793b9906de0294663 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 484e26ab9179442793b9906de0294663; no voters:
I20260809 02:02:28.692873 3041 leader_election.cc:288] T 00000000000000000000000000000000 P 484e26ab9179442793b9906de0294663 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:02:28.692945 3046 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 484e26ab9179442793b9906de0294663 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:02:28.693171 3046 raft_consensus.cc:686] T 00000000000000000000000000000000 P 484e26ab9179442793b9906de0294663 [term 1 LEADER]: Becoming Leader. State: Replica: 484e26ab9179442793b9906de0294663, State: Running, Role: LEADER
I20260809 02:02:28.693528 3046 consensus_queue.cc:237] T 00000000000000000000000000000000 P 484e26ab9179442793b9906de0294663 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "484e26ab9179442793b9906de0294663" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 39235 } }
I20260809 02:02:28.694779 3044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:28.680149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43798 (local address 127.1.192.254:39235)
0809 02:02:28.680620 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:28.680647 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:28.680688 (+ 41us) server_negotiation.cc:407] Connection header received
0809 02:02:28.681422 (+ 734us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:28.681440 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:28.681836 (+ 396us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:28.682143 (+ 307us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:28.683176 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:28.684387 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:28.685370 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:28.685969 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:28.686192 (+ 223us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:28.688960 (+ 2768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:02:28.688968 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:02:28.688987 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:02:28.689018 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:02:28.691341 (+ 2323us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:28.691828 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:28.691834 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:28.691840 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:28.691997 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:28.692371 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:28.692395 (+ 24us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:28.692400 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:28.692970 (+ 570us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:02:28.693223 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:28.693523 (+ 300us) server_negotiation.cc:299] Negotiation successful
0809 02:02:28.693872 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":160,"threads_started":1}
I20260809 02:02:28.694751 3046 sys_catalog.cc:455] T 00000000000000000000000000000000 P 484e26ab9179442793b9906de0294663 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "484e26ab9179442793b9906de0294663" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "484e26ab9179442793b9906de0294663" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 39235 } } }
I20260809 02:02:28.694891 3041 sys_catalog.cc:564] T 00000000000000000000000000000000 P 484e26ab9179442793b9906de0294663 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:02:28.694891 3046 sys_catalog.cc:458] T 00000000000000000000000000000000 P 484e26ab9179442793b9906de0294663 [sys.catalog]: This master's current role is: LEADER
I20260809 02:02:28.695302 3050 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:02:28.695479 3047 sys_catalog.cc:455] T 00000000000000000000000000000000 P 484e26ab9179442793b9906de0294663 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 484e26ab9179442793b9906de0294663. Latest consensus state: current_term: 1 leader_uuid: "484e26ab9179442793b9906de0294663" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "484e26ab9179442793b9906de0294663" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 39235 } } }
I20260809 02:02:28.695647 3047 sys_catalog.cc:458] T 00000000000000000000000000000000 P 484e26ab9179442793b9906de0294663 [sys.catalog]: This master's current role is: LEADER
I20260809 02:02:28.698521 3050 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:02:28.704432 3050 catalog_manager.cc:1347] Generated new cluster ID: a58970ee2b9842b39bfbfacf3026d34b
I20260809 02:02:28.704524 3050 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:02:28.711359 3050 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:02:28.712388 3050 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:02:28.721086 3050 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 484e26ab9179442793b9906de0294663: Generated new TSK 0
I20260809 02:02:28.721928 3050 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.192.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.192.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.192.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.193.keytab.
Entry for principal kudu/127.1.192.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.193.keytab.
I20260809 02:02:28.757629 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786240941968412-1795-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786240941968412-1795-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786240941968412-1795-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786240941968412-1795-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.192.193:0
--local_ip_for_outbound_sockets=127.1.192.193
--webserver_interface=127.1.192.193
--webserver_port=0
--tserver_master_addrs=127.1.192.254:39235
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.193.keytab
--principal=kudu/127.1.192.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:43057
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:02:28.903791 3067 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:02:28.904109 3067 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:02:28.904192 3067 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:02:28.910953 3067 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:02:28.911051 3067 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:02:28.911123 3067 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 02:02:28.911190 3067 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.192.193
I20260809 02:02:28.916924 3067 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:43057
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786240941968412-1795-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786240941968412-1795-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.193.keytab
--principal=kudu/127.1.192.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.192.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786240941968412-1795-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.192.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.192.254:39235
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3067
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.192.193
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786240941968412-1795-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:02:28.918183 3067 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:02:28.919263 3067 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:02:28.929659 3073 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:02:28.929627 3077 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:02:28.929626 3074 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:02:28.929924 3067 server_base.cc:1034] running on GCE node
I20260809 02:02:28.930482 3067 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:02:28.931260 3067 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:02:28.932468 3067 hybrid_clock.cc:648] HybridClock initialized: now 1786240948932439 us; error 49 us; skew 500 ppm
Aug 09 02:02:28 dist-test-slave-ss4n krb5kdc[2943](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240948, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:02:28.941867 3067 init.cc:377] Logged in from keytab as kudu/127.1.192.193@KRBTEST.COM (short username kudu)
I20260809 02:02:28.943032 3067 webserver.cc:460] Webserver started at http://127.1.192.193:43895/ using document root <none> and password file <none>
I20260809 02:02:28.943719 3067 fs_manager.cc:359] Metadata directory not provided
I20260809 02:02:28.943838 3067 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:02:28.944137 3067 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:02:28.946933 3067 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786240941968412-1795-0/minicluster-data/ts-0/data/instance:
uuid: "350e9b1bbad84279bf35b4e4db434b6b"
format_stamp: "Formatted at 2026-08-09 02:02:28 on dist-test-slave-ss4n"
server_key: "9847825f619eff37cb42e4fc432ae35c"
server_key_iv: "b2124f125a4ba35dde3d76b7f7f5d655"
server_key_version: "encryptionkey@0"
I20260809 02:02:28.947921 3067 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786240941968412-1795-0/minicluster-data/ts-0/wal/instance:
uuid: "350e9b1bbad84279bf35b4e4db434b6b"
format_stamp: "Formatted at 2026-08-09 02:02:28 on dist-test-slave-ss4n"
server_key: "9847825f619eff37cb42e4fc432ae35c"
server_key_iv: "b2124f125a4ba35dde3d76b7f7f5d655"
server_key_version: "encryptionkey@0"
I20260809 02:02:28.952459 3067 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 02:02:28.955564 3084 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:28.957012 3067 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 02:02:28.957206 3067 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786240941968412-1795-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786240941968412-1795-0/minicluster-data/ts-0/wal
uuid: "350e9b1bbad84279bf35b4e4db434b6b"
format_stamp: "Formatted at 2026-08-09 02:02:28 on dist-test-slave-ss4n"
server_key: "9847825f619eff37cb42e4fc432ae35c"
server_key_iv: "b2124f125a4ba35dde3d76b7f7f5d655"
server_key_version: "encryptionkey@0"
I20260809 02:02:28.957346 3067 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786240941968412-1795-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786240941968412-1795-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786240941968412-1795-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:02:28.975324 3067 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:02:28.978714 3067 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:02:28.978957 3067 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:02:28.980571 3067 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:02:28.980660 3067 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:28.980736 3067 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:02:28.980791 3067 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 02:02:28 dist-test-slave-ss4n krb5kdc[2943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240948, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.193@KRBTEST.COM for kudu/127.1.192.254@KRBTEST.COM
I20260809 02:02:28.992837 3067 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.193:40369
I20260809 02:02:28.992905 3207 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.193:40369 every 8 connection(s)
I20260809 02:02:28.994094 3067 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786240941968412-1795-0/minicluster-data/ts-0/data/info.pb
I20260809 02:02:28.995895 3044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:28.981350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.193:47213 (local address 127.1.192.254:39235)
0809 02:02:28.981562 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:28.981570 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:28.982440 (+ 870us) server_negotiation.cc:407] Connection header received
0809 02:02:28.983426 (+ 986us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:28.983434 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:28.983510 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:28.983633 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:28.985221 (+ 1588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:28.986286 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:28.987521 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:28.987764 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:28.987880 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:28.991533 (+ 3653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:02:28.991537 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:02:28.991539 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:02:28.991575 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:02:28.993312 (+ 1737us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:28.993979 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:28.993982 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:28.993983 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:28.994062 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:28.994466 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:28.994468 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:28.994469 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:28.994806 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:02:28.994909 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:28.995643 (+ 734us) server_negotiation.cc:299] Negotiation successful
0809 02:02:28.995769 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 02:02:28.996835 3093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:28.981792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:39235 (local address 127.1.192.193:47213)
0809 02:02:28.982259 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:28.982310 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 02:02:28.983212 (+ 902us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:02:28.983826 (+ 614us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:02:28.983836 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:02:28.984299 (+ 463us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:02:28.985028 (+ 729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:28.985045 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:28.986442 (+ 1397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:28.986446 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:28.987372 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:28.987382 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:28.987928 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:28.987932 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:28.988143 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:28.988809 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:02:28.988839 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:02:28.991279 (+ 2440us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:02:28.993439 (+ 2160us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:28.993445 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:28.993461 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:28.993867 (+ 406us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:28.994175 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:28.994179 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:28.994181 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:28.994359 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:28.994964 (+ 605us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:02:28.994971 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:02:28.995328 (+ 357us) client_negotiation.cc:769] Sending connection context
0809 02:02:28.995489 (+ 161us) client_negotiation.cc:240] Negotiation successful
0809 02:02:28.995801 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":124,"threads_started":1}
I20260809 02:02:28.998450 3208 heartbeater.cc:347] Connected to a master server at 127.1.192.254:39235
I20260809 02:02:28.998618 3208 heartbeater.cc:464] Registering TS with master...
I20260809 02:02:28.999105 3208 heartbeater.cc:511] Master 127.1.192.254:39235 requested a full tablet report, sending...
I20260809 02:02:28.999543 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 3067
I20260809 02:02:28.999684 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786240941968412-1795-0/minicluster-data/ts-0/wal/instance
I20260809 02:02:28.999998 1795 external_mini_cluster.cc:1428] Setting key b26da8754bb4d51de168ced66900c976
I20260809 02:02:29.001677 2991 ts_manager.cc:194] Registered new tserver with Master: 350e9b1bbad84279bf35b4e4db434b6b (127.1.192.193:40369)
I20260809 02:02:29.003216 2991 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.192.193@KRBTEST.COM'} at 127.1.192.193:47213
I20260809 02:02:29.008860 3044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:29.000976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.193:34343 (local address 127.1.192.254:39235)
0809 02:02:29.001110 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:29.001115 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:29.001343 (+ 228us) server_negotiation.cc:407] Connection header received
0809 02:02:29.001445 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:29.001447 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:29.001557 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:29.001656 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:29.002201 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:29.002775 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:29.003788 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:29.004151 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:29.004267 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:29.005626 (+ 1359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:02:29.005637 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:02:29.005639 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:02:29.005664 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:02:29.007188 (+ 1524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:29.007650 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:29.007653 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:29.007655 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:29.007733 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:29.008011 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:29.008016 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:29.008018 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:29.008194 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:02:29.008305 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:29.008570 (+ 265us) server_negotiation.cc:299] Negotiation successful
0809 02:02:29.008698 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 02:02:29.008868 3217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:29.000910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:39235 (local address 127.1.192.193:34343)
0809 02:02:29.001237 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:29.001254 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 02:02:29.001357 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:02:29.001692 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:02:29.001698 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:02:29.001924 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:02:29.002059 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:29.002067 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:29.002917 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:29.002921 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:29.003660 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:29.003669 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:29.004294 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:29.004301 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:29.004436 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:29.005081 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:02:29.005093 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:02:29.005451 (+ 358us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:02:29.007330 (+ 1879us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:29.007335 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:29.007337 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:29.007539 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:29.007827 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:29.007832 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:29.007834 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:29.007929 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:29.008295 (+ 366us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:02:29.008298 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:02:29.008397 (+ 99us) client_negotiation.cc:769] Sending connection context
0809 02:02:29.008504 (+ 107us) client_negotiation.cc:240] Negotiation successful
0809 02:02:29.008658 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":126,"threads_started":1}
WARNING: no policy specified for kudu/127.1.192.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.192.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.192.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.194.keytab.
Entry for principal kudu/127.1.192.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.194.keytab.
I20260809 02:02:29.037871 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786240941968412-1795-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786240941968412-1795-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786240941968412-1795-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786240941968412-1795-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.192.194:0
--local_ip_for_outbound_sockets=127.1.192.194
--webserver_interface=127.1.192.194
--webserver_port=0
--tserver_master_addrs=127.1.192.254:39235
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.194.keytab
--principal=kudu/127.1.192.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:43057
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:02:29.152168 3220 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:02:29.152454 3220 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:02:29.152500 3220 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:02:29.156514 3220 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:02:29.156580 3220 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:02:29.156620 3220 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 02:02:29.156653 3220 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.192.194
I20260809 02:02:29.162114 3220 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:43057
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786240941968412-1795-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786240941968412-1795-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.194.keytab
--principal=kudu/127.1.192.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.192.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786240941968412-1795-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.192.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.192.254:39235
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3220
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.192.194
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786240941968412-1795-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:02:29.163872 3220 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:02:29.164950 3220 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:02:29.174619 3228 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:02:29.174780 3226 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:02:29.174875 3220 server_base.cc:1034] running on GCE node
W20260809 02:02:29.174639 3225 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:02:29.175308 3220 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:02:29.175973 3220 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:02:29.177178 3220 hybrid_clock.cc:648] HybridClock initialized: now 1786240949177161 us; error 28 us; skew 500 ppm
Aug 09 02:02:29 dist-test-slave-ss4n krb5kdc[2943](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240949, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:02:29.180361 3220 init.cc:377] Logged in from keytab as kudu/127.1.192.194@KRBTEST.COM (short username kudu)
I20260809 02:02:29.181391 3220 webserver.cc:460] Webserver started at http://127.1.192.194:38951/ using document root <none> and password file <none>
I20260809 02:02:29.182029 3220 fs_manager.cc:359] Metadata directory not provided
I20260809 02:02:29.182106 3220 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:02:29.182325 3220 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:02:29.184481 3220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786240941968412-1795-0/minicluster-data/ts-1/data/instance:
uuid: "374d1f9d096c4e89b211c84c4228b9ae"
format_stamp: "Formatted at 2026-08-09 02:02:29 on dist-test-slave-ss4n"
server_key: "93be5b2356a28a65d6b91f63a31e413b"
server_key_iv: "53485e2b8b9e910071977d61c660acd6"
server_key_version: "encryptionkey@0"
I20260809 02:02:29.185176 3220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786240941968412-1795-0/minicluster-data/ts-1/wal/instance:
uuid: "374d1f9d096c4e89b211c84c4228b9ae"
format_stamp: "Formatted at 2026-08-09 02:02:29 on dist-test-slave-ss4n"
server_key: "93be5b2356a28a65d6b91f63a31e413b"
server_key_iv: "53485e2b8b9e910071977d61c660acd6"
server_key_version: "encryptionkey@0"
I20260809 02:02:29.189185 3220 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260809 02:02:29.191865 3236 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:29.193197 3220 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 02:02:29.193356 3220 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786240941968412-1795-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786240941968412-1795-0/minicluster-data/ts-1/wal
uuid: "374d1f9d096c4e89b211c84c4228b9ae"
format_stamp: "Formatted at 2026-08-09 02:02:29 on dist-test-slave-ss4n"
server_key: "93be5b2356a28a65d6b91f63a31e413b"
server_key_iv: "53485e2b8b9e910071977d61c660acd6"
server_key_version: "encryptionkey@0"
I20260809 02:02:29.193552 3220 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786240941968412-1795-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786240941968412-1795-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786240941968412-1795-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:02:29.219291 3220 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:02:29.223011 3220 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:02:29.223227 3220 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:02:29.224740 3220 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:02:29.224792 3220 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:29.224833 3220 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:02:29.224905 3220 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 02:02:29 dist-test-slave-ss4n krb5kdc[2943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240949, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.194@KRBTEST.COM for kudu/127.1.192.254@KRBTEST.COM
I20260809 02:02:29.236526 3220 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.194:40381
I20260809 02:02:29.237402 3220 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786240941968412-1795-0/minicluster-data/ts-1/data/info.pb
I20260809 02:02:29.238260 3363 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.194:40381 every 8 connection(s)
I20260809 02:02:29.239171 3044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:29.225744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.194:42689 (local address 127.1.192.254:39235)
0809 02:02:29.225880 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:29.225884 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:29.226960 (+ 1076us) server_negotiation.cc:407] Connection header received
0809 02:02:29.228107 (+ 1147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:29.228111 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:29.228170 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:29.228278 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:29.229444 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:29.230249 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:29.231693 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:29.232040 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:29.232161 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:29.235306 (+ 3145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:02:29.235315 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:02:29.235317 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:02:29.235343 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:02:29.237023 (+ 1680us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:29.237600 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:29.237604 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:29.237606 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:29.237696 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:29.238105 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:29.238108 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:29.238111 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:29.238375 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:02:29.238481 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:29.238936 (+ 455us) server_negotiation.cc:299] Negotiation successful
0809 02:02:29.239047 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 02:02:29.239967 3245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:29.226310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:39235 (local address 127.1.192.194:42689)
0809 02:02:29.226813 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:29.226855 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 02:02:29.227867 (+ 1012us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:02:29.228453 (+ 586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:02:29.228459 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:02:29.228807 (+ 348us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:02:29.229291 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:29.229304 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:29.230377 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:29.230381 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:29.231552 (+ 1171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:29.231561 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:29.232218 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:29.232222 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:29.232400 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:29.233064 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:02:29.233091 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:02:29.235101 (+ 2010us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:02:29.237173 (+ 2072us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:29.237179 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:29.237194 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:29.237490 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:29.237794 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:29.237799 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:29.237801 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:29.237984 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:29.238472 (+ 488us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:02:29.238478 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:02:29.238711 (+ 233us) client_negotiation.cc:769] Sending connection context
0809 02:02:29.238863 (+ 152us) client_negotiation.cc:240] Negotiation successful
0809 02:02:29.239121 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":145,"threads_started":1}
I20260809 02:02:29.241343 3364 heartbeater.cc:347] Connected to a master server at 127.1.192.254:39235
I20260809 02:02:29.241665 3364 heartbeater.cc:464] Registering TS with master...
I20260809 02:02:29.242219 3364 heartbeater.cc:511] Master 127.1.192.254:39235 requested a full tablet report, sending...
I20260809 02:02:29.243371 2987 ts_manager.cc:194] Registered new tserver with Master: 374d1f9d096c4e89b211c84c4228b9ae (127.1.192.194:40381)
I20260809 02:02:29.244104 2987 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.192.194@KRBTEST.COM'} at 127.1.192.194:42689
I20260809 02:02:29.245833 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 3220
I20260809 02:02:29.245992 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786240941968412-1795-0/minicluster-data/ts-1/wal/instance
I20260809 02:02:29.246330 1795 external_mini_cluster.cc:1428] Setting key b99471097c88a04ffc93354989346b11
I20260809 02:02:29.252351 3044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:29.243944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.194:33995 (local address 127.1.192.254:39235)
0809 02:02:29.244073 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:29.244078 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:29.244360 (+ 282us) server_negotiation.cc:407] Connection header received
0809 02:02:29.244459 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:29.244463 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:29.244525 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:29.244603 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:29.245101 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:29.246003 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:29.247182 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:29.247451 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:29.247611 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:29.248868 (+ 1257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:02:29.248877 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:02:29.248880 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:02:29.248902 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:02:29.250606 (+ 1704us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:29.251070 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:29.251073 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:29.251075 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:29.251182 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:29.251471 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:29.251476 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:29.251478 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:29.251704 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:02:29.251820 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:29.252092 (+ 272us) server_negotiation.cc:299] Negotiation successful
0809 02:02:29.252205 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":17,"server-negotiator.queue_time_us":53}
I20260809 02:02:29.252364 3371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:29.243814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:39235 (local address 127.1.192.194:33995)
0809 02:02:29.244241 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:29.244256 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 02:02:29.244362 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:02:29.244650 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:02:29.244656 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:02:29.244862 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:02:29.244983 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:29.244989 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:29.246177 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:29.246181 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:29.247051 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:29.247062 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:29.247591 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:29.247595 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:29.247723 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:29.248311 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:02:29.248331 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:02:29.248704 (+ 373us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:02:29.250743 (+ 2039us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:29.250747 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:29.250749 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:29.250956 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:29.251285 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:29.251288 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:29.251290 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:29.251378 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:29.251808 (+ 430us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:02:29.251811 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:02:29.251910 (+ 99us) client_negotiation.cc:769] Sending connection context
0809 02:02:29.252018 (+ 108us) client_negotiation.cc:240] Negotiation successful
0809 02:02:29.252169 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":340,"mutex_wait_us":2,"thread_start_us":123,"threads_started":1}
WARNING: no policy specified for kudu/127.1.192.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.192.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.192.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.195.keytab.
Entry for principal kudu/127.1.192.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.195.keytab.
I20260809 02:02:29.280920 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786240941968412-1795-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786240941968412-1795-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786240941968412-1795-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786240941968412-1795-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.192.195:0
--local_ip_for_outbound_sockets=127.1.192.195
--webserver_interface=127.1.192.195
--webserver_port=0
--tserver_master_addrs=127.1.192.254:39235
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.195.keytab
--principal=kudu/127.1.192.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:43057
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:02:29.406935 3374 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:02:29.407285 3374 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:02:29.407383 3374 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:02:29.412014 3374 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:02:29.412283 3374 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:02:29.412426 3374 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 02:02:29.412513 3374 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.192.195
I20260809 02:02:29.418915 3374 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:43057
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786240941968412-1795-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786240941968412-1795-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.195.keytab
--principal=kudu/127.1.192.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.192.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786240941968412-1795-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.192.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.192.254:39235
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3374
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.192.195
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786240941968412-1795-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:02:29.420096 3374 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:02:29.421167 3374 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:02:29.433038 3383 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:02:29.433025 3381 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:02:29.433032 3380 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:02:29.433516 3374 server_base.cc:1034] running on GCE node
I20260809 02:02:29.434011 3374 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:02:29.434737 3374 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:02:29.435993 3374 hybrid_clock.cc:648] HybridClock initialized: now 1786240949435977 us; error 42 us; skew 500 ppm
Aug 09 02:02:29 dist-test-slave-ss4n krb5kdc[2943](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240949, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:02:29.439394 3374 init.cc:377] Logged in from keytab as kudu/127.1.192.195@KRBTEST.COM (short username kudu)
I20260809 02:02:29.440433 3374 webserver.cc:460] Webserver started at http://127.1.192.195:38323/ using document root <none> and password file <none>
I20260809 02:02:29.441025 3374 fs_manager.cc:359] Metadata directory not provided
I20260809 02:02:29.441089 3374 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:02:29.441275 3374 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:02:29.443655 3374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786240941968412-1795-0/minicluster-data/ts-2/data/instance:
uuid: "905d1a29a3c4431499499b3df47ecaf4"
format_stamp: "Formatted at 2026-08-09 02:02:29 on dist-test-slave-ss4n"
server_key: "8204e5954285e06ebf3c3ea6e3a41b0d"
server_key_iv: "faa87ac0be72116cea1d5bc2de4b88ad"
server_key_version: "encryptionkey@0"
I20260809 02:02:29.444228 3374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786240941968412-1795-0/minicluster-data/ts-2/wal/instance:
uuid: "905d1a29a3c4431499499b3df47ecaf4"
format_stamp: "Formatted at 2026-08-09 02:02:29 on dist-test-slave-ss4n"
server_key: "8204e5954285e06ebf3c3ea6e3a41b0d"
server_key_iv: "faa87ac0be72116cea1d5bc2de4b88ad"
server_key_version: "encryptionkey@0"
I20260809 02:02:29.448455 3374 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 02:02:29.451297 3390 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:29.452694 3374 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 02:02:29.452839 3374 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786240941968412-1795-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786240941968412-1795-0/minicluster-data/ts-2/wal
uuid: "905d1a29a3c4431499499b3df47ecaf4"
format_stamp: "Formatted at 2026-08-09 02:02:29 on dist-test-slave-ss4n"
server_key: "8204e5954285e06ebf3c3ea6e3a41b0d"
server_key_iv: "faa87ac0be72116cea1d5bc2de4b88ad"
server_key_version: "encryptionkey@0"
I20260809 02:02:29.452944 3374 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786240941968412-1795-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786240941968412-1795-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786240941968412-1795-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:02:29.482951 3374 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:02:29.486462 3374 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:02:29.486642 3374 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:02:29.488173 3374 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:02:29.488231 3374 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:29.488272 3374 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:02:29.488288 3374 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 02:02:29 dist-test-slave-ss4n krb5kdc[2943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240949, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.195@KRBTEST.COM for kudu/127.1.192.254@KRBTEST.COM
I20260809 02:02:29.500072 3374 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.195:42657
I20260809 02:02:29.500139 3508 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.195:42657 every 8 connection(s)
I20260809 02:02:29.501528 3374 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786240941968412-1795-0/minicluster-data/ts-2/data/info.pb
I20260809 02:02:29.504997 3044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:29.489222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.195:39849 (local address 127.1.192.254:39235)
0809 02:02:29.489396 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:29.489401 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:29.490486 (+ 1085us) server_negotiation.cc:407] Connection header received
0809 02:02:29.491831 (+ 1345us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:29.491835 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:29.491902 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:29.492122 (+ 220us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:29.494011 (+ 1889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:29.494822 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:29.496351 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:29.496731 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:29.496871 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:29.500686 (+ 3815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:02:29.500691 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:02:29.500694 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:02:29.500732 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:02:29.502702 (+ 1970us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:29.503355 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:29.503358 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:29.503360 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:29.503458 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:29.503833 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:29.503836 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:29.503838 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:29.504048 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:02:29.504171 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:29.504686 (+ 515us) server_negotiation.cc:299] Negotiation successful
0809 02:02:29.504821 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 02:02:29.505878 3399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:29.489726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:39235 (local address 127.1.192.195:39849)
0809 02:02:29.490315 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:29.490370 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0809 02:02:29.491587 (+ 1217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:02:29.492405 (+ 818us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:02:29.492414 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:02:29.492974 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:02:29.493801 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:29.493827 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:29.494982 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:29.494986 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:29.496188 (+ 1202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:29.496199 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:29.496929 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:29.496934 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:29.497135 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:29.498105 (+ 970us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:02:29.498130 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:02:29.500467 (+ 2337us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:02:29.502841 (+ 2374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:29.502847 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:29.502862 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:29.503248 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:29.503563 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:29.503566 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:29.503568 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:29.503720 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:29.504166 (+ 446us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:02:29.504173 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:02:29.504430 (+ 257us) client_negotiation.cc:769] Sending connection context
0809 02:02:29.504621 (+ 191us) client_negotiation.cc:240] Negotiation successful
0809 02:02:29.504959 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":235,"threads_started":1}
I20260809 02:02:29.507247 3509 heartbeater.cc:347] Connected to a master server at 127.1.192.254:39235
I20260809 02:02:29.507397 3509 heartbeater.cc:464] Registering TS with master...
I20260809 02:02:29.507997 3509 heartbeater.cc:511] Master 127.1.192.254:39235 requested a full tablet report, sending...
I20260809 02:02:29.509218 2991 ts_manager.cc:194] Registered new tserver with Master: 905d1a29a3c4431499499b3df47ecaf4 (127.1.192.195:42657)
I20260809 02:02:29.509483 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 3374
I20260809 02:02:29.509687 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786240941968412-1795-0/minicluster-data/ts-2/wal/instance
I20260809 02:02:29.510049 1795 external_mini_cluster.cc:1428] Setting key a82ecfbf68afca449516148cc98e3127
I20260809 02:02:29.510061 2991 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.192.195@KRBTEST.COM'} at 127.1.192.195:39849
I20260809 02:02:29.512926 1795 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 02:02:29.518875 3044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:29.510401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.195:37929 (local address 127.1.192.254:39235)
0809 02:02:29.510563 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:29.510570 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:29.510585 (+ 15us) server_negotiation.cc:407] Connection header received
0809 02:02:29.510625 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:29.510628 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:29.510675 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:29.510761 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:29.511447 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:29.511968 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:29.513089 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:29.513411 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:29.513570 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:29.514953 (+ 1383us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:02:29.514959 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:02:29.514965 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:02:29.515000 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:02:29.516783 (+ 1783us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:29.517362 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:29.517365 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:29.517366 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:29.517453 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:29.517914 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:29.517917 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:29.517919 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:29.518164 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:02:29.518268 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:29.518616 (+ 348us) server_negotiation.cc:299] Negotiation successful
0809 02:02:29.518738 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 02:02:29.519057 3515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:29.509782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:39235 (local address 127.1.192.195:37929)
0809 02:02:29.510139 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:29.510156 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 02:02:29.510264 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:02:29.510911 (+ 647us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:02:29.510914 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:02:29.511195 (+ 281us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:02:29.511325 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:29.511334 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:29.512118 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:29.512122 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:29.512925 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:29.512945 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:29.513571 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:29.513574 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:29.513670 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:29.514307 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:02:29.514320 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:02:29.514790 (+ 470us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:02:29.516927 (+ 2137us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:29.516930 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:29.516933 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:29.517209 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:29.517662 (+ 453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:29.517668 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:29.517673 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:29.517784 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:29.518287 (+ 503us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:02:29.518293 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:02:29.518422 (+ 129us) client_negotiation.cc:769] Sending connection context
0809 02:02:29.518542 (+ 120us) client_negotiation.cc:240] Negotiation successful
0809 02:02:29.518751 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"mutex_wait_us":1,"thread_start_us":139,"threads_started":1}
Aug 09 02:02:29 dist-test-slave-ss4n krb5kdc[2943](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240949, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 02:02:29 dist-test-slave-ss4n krb5kdc[2943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240949, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.192.254@KRBTEST.COM
I20260809 02:02:29.546770 3044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:29.535920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43800 (local address 127.1.192.254:39235)
0809 02:02:29.536103 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:29.536109 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:29.536122 (+ 13us) server_negotiation.cc:407] Connection header received
0809 02:02:29.536184 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:29.536187 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:29.536234 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:29.536342 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:29.537164 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:29.537830 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:29.538803 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:29.539078 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:29.539213 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:29.542629 (+ 3416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:02:29.542633 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:02:29.542635 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:02:29.542667 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:02:29.544670 (+ 2003us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:29.545244 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:29.545248 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:29.545251 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:29.545352 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:29.545739 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:29.545744 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:29.545746 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:29.545970 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:02:29.546145 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:29.546426 (+ 281us) server_negotiation.cc:299] Negotiation successful
0809 02:02:29.546578 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 02:02:29.550330 2991 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43800:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 02:02:29.553771 2991 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:02:29.569517 3531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:29.563550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.194:40381 (local address 127.0.0.1:45016)
0809 02:02:29.564356 (+ 806us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:29.564371 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 02:02:29.564460 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:02:29.565244 (+ 784us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:02:29.565247 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:02:29.565265 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:02:29.565375 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:29.565384 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:29.566742 (+ 1358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:29.566748 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:29.567676 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:29.567685 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:29.568867 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:29.568871 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:29.569009 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:29.569054 (+ 45us) client_negotiation.cc:769] Sending connection context
0809 02:02:29.569187 (+ 133us) client_negotiation.cc:240] Negotiation successful
0809 02:02:29.569312 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":743,"mutex_wait_us":23,"thread_start_us":132,"threads_started":1}
I20260809 02:02:29.569571 3533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:29.564072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45016 (local address 127.1.192.194:40381)
0809 02:02:29.564854 (+ 782us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:29.564866 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:29.564888 (+ 22us) server_negotiation.cc:407] Connection header received
0809 02:02:29.564970 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:29.564974 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:29.565105 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:29.565266 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:02:29.565550 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:29.566627 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:29.567810 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:29.568719 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:29.568849 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:29.569059 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:29.569248 (+ 189us) server_negotiation.cc:299] Negotiation successful
0809 02:02:29.569376 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":674,"thread_start_us":185,"threads_started":1}
I20260809 02:02:29.569976 3527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:29.563314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.193:40369 (local address 127.0.0.1:45388)
0809 02:02:29.563866 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:29.563922 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0809 02:02:29.564054 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:02:29.564824 (+ 770us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:02:29.564830 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:02:29.564855 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:02:29.565031 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:29.565037 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:29.566549 (+ 1512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:29.566552 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:29.568037 (+ 1485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:29.568049 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:29.569356 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:29.569359 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:29.569466 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:29.569521 (+ 55us) client_negotiation.cc:769] Sending connection context
0809 02:02:29.569578 (+ 57us) client_negotiation.cc:240] Negotiation successful
0809 02:02:29.569814 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":381,"mutex_wait_us":76,"thread_start_us":133,"threads_started":1}
I20260809 02:02:29.570506 3528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:29.563429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45388 (local address 127.1.192.193:40369)
0809 02:02:29.564279 (+ 850us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:29.564286 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:29.564310 (+ 24us) server_negotiation.cc:407] Connection header received
0809 02:02:29.564393 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:29.564398 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:29.564625 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:29.565166 (+ 541us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:02:29.565273 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:29.566424 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:29.568271 (+ 1847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:29.569209 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:29.570032 (+ 823us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:29.570206 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:29.570259 (+ 53us) server_negotiation.cc:299] Negotiation successful
0809 02:02:29.570353 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":736,"thread_start_us":77,"threads_started":1}
I20260809 02:02:29.572129 3530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:29.564075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.195:42657 (local address 127.0.0.1:60718)
0809 02:02:29.565530 (+ 1455us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:29.565544 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 02:02:29.565628 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:02:29.565993 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:02:29.565996 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:02:29.566013 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:02:29.566109 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:29.566128 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:29.568402 (+ 2274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:29.568406 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:29.569619 (+ 1213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:29.569836 (+ 217us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:29.571692 (+ 1856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:29.571695 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:29.571783 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:29.571804 (+ 21us) client_negotiation.cc:769] Sending connection context
0809 02:02:29.571902 (+ 98us) client_negotiation.cc:240] Negotiation successful
0809 02:02:29.571982 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1350,"mutex_wait_us":1,"thread_start_us":141,"threads_started":1}
I20260809 02:02:29.572266 3534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:29.564523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60718 (local address 127.1.192.195:42657)
0809 02:02:29.564947 (+ 424us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:29.564957 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:29.565627 (+ 670us) server_negotiation.cc:407] Connection header received
0809 02:02:29.565736 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:29.565739 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:29.565867 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:29.566015 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:02:29.566215 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:29.568275 (+ 2060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:29.570697 (+ 2422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:29.571511 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:29.571629 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:29.571772 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:29.571956 (+ 184us) server_negotiation.cc:299] Negotiation successful
0809 02:02:29.572057 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":330,"thread_start_us":115,"threads_started":1}
I20260809 02:02:29.573665 3291 tablet_service.cc:1467] Processing CreateTablet for tablet 76aaf83ff0a94d3c9994c8eda3457f17 (DEFAULT_TABLE table=test-table [id=4c7442b6c12341888d847bfab0b57582]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:02:29.573976 3136 tablet_service.cc:1467] Processing CreateTablet for tablet 76aaf83ff0a94d3c9994c8eda3457f17 (DEFAULT_TABLE table=test-table [id=4c7442b6c12341888d847bfab0b57582]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:02:29.574887 3136 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 76aaf83ff0a94d3c9994c8eda3457f17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:02:29.574887 3291 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 76aaf83ff0a94d3c9994c8eda3457f17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:02:29.574877 3440 tablet_service.cc:1467] Processing CreateTablet for tablet 76aaf83ff0a94d3c9994c8eda3457f17 (DEFAULT_TABLE table=test-table [id=4c7442b6c12341888d847bfab0b57582]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:02:29.575870 3440 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 76aaf83ff0a94d3c9994c8eda3457f17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:02:29.582666 3535 tablet_bootstrap.cc:492] T 76aaf83ff0a94d3c9994c8eda3457f17 P 350e9b1bbad84279bf35b4e4db434b6b: Bootstrap starting.
I20260809 02:02:29.583474 3538 tablet_bootstrap.cc:492] T 76aaf83ff0a94d3c9994c8eda3457f17 P 374d1f9d096c4e89b211c84c4228b9ae: Bootstrap starting.
I20260809 02:02:29.583647 3536 tablet_bootstrap.cc:492] T 76aaf83ff0a94d3c9994c8eda3457f17 P 905d1a29a3c4431499499b3df47ecaf4: Bootstrap starting.
I20260809 02:02:29.585394 3538 tablet_bootstrap.cc:654] T 76aaf83ff0a94d3c9994c8eda3457f17 P 374d1f9d096c4e89b211c84c4228b9ae: Neither blocks nor log segments found. Creating new log.
I20260809 02:02:29.585587 3536 tablet_bootstrap.cc:654] T 76aaf83ff0a94d3c9994c8eda3457f17 P 905d1a29a3c4431499499b3df47ecaf4: Neither blocks nor log segments found. Creating new log.
I20260809 02:02:29.586372 3538 log.cc:824] T 76aaf83ff0a94d3c9994c8eda3457f17 P 374d1f9d096c4e89b211c84c4228b9ae: Log is configured to *not* fsync() on all Append() calls
I20260809 02:02:29.586489 3536 log.cc:824] T 76aaf83ff0a94d3c9994c8eda3457f17 P 905d1a29a3c4431499499b3df47ecaf4: Log is configured to *not* fsync() on all Append() calls
I20260809 02:02:29.588174 3538 tablet_bootstrap.cc:492] T 76aaf83ff0a94d3c9994c8eda3457f17 P 374d1f9d096c4e89b211c84c4228b9ae: No bootstrap required, opened a new log
I20260809 02:02:29.588375 3538 ts_tablet_manager.cc:1397] T 76aaf83ff0a94d3c9994c8eda3457f17 P 374d1f9d096c4e89b211c84c4228b9ae: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260809 02:02:29.588851 3535 tablet_bootstrap.cc:654] T 76aaf83ff0a94d3c9994c8eda3457f17 P 350e9b1bbad84279bf35b4e4db434b6b: Neither blocks nor log segments found. Creating new log.
I20260809 02:02:29.589645 3535 log.cc:824] T 76aaf83ff0a94d3c9994c8eda3457f17 P 350e9b1bbad84279bf35b4e4db434b6b: Log is configured to *not* fsync() on all Append() calls
I20260809 02:02:29.591311 3536 tablet_bootstrap.cc:492] T 76aaf83ff0a94d3c9994c8eda3457f17 P 905d1a29a3c4431499499b3df47ecaf4: No bootstrap required, opened a new log
I20260809 02:02:29.591538 3536 ts_tablet_manager.cc:1397] T 76aaf83ff0a94d3c9994c8eda3457f17 P 905d1a29a3c4431499499b3df47ecaf4: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.000s
I20260809 02:02:29.591665 3535 tablet_bootstrap.cc:492] T 76aaf83ff0a94d3c9994c8eda3457f17 P 350e9b1bbad84279bf35b4e4db434b6b: No bootstrap required, opened a new log
I20260809 02:02:29.591850 3535 ts_tablet_manager.cc:1397] T 76aaf83ff0a94d3c9994c8eda3457f17 P 350e9b1bbad84279bf35b4e4db434b6b: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.003s
I20260809 02:02:29.592075 3538 raft_consensus.cc:348] T 76aaf83ff0a94d3c9994c8eda3457f17 P 374d1f9d096c4e89b211c84c4228b9ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "350e9b1bbad84279bf35b4e4db434b6b" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 40369 } } peers { permanent_uuid: "905d1a29a3c4431499499b3df47ecaf4" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 42657 } } peers { permanent_uuid: "374d1f9d096c4e89b211c84c4228b9ae" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 40381 } }
I20260809 02:02:29.592360 3538 raft_consensus.cc:374] T 76aaf83ff0a94d3c9994c8eda3457f17 P 374d1f9d096c4e89b211c84c4228b9ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:02:29.592412 3538 raft_consensus.cc:729] T 76aaf83ff0a94d3c9994c8eda3457f17 P 374d1f9d096c4e89b211c84c4228b9ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 374d1f9d096c4e89b211c84c4228b9ae, State: Initialized, Role: FOLLOWER
I20260809 02:02:29.593096 3538 consensus_queue.cc:260] T 76aaf83ff0a94d3c9994c8eda3457f17 P 374d1f9d096c4e89b211c84c4228b9ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "350e9b1bbad84279bf35b4e4db434b6b" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 40369 } } peers { permanent_uuid: "905d1a29a3c4431499499b3df47ecaf4" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 42657 } } peers { permanent_uuid: "374d1f9d096c4e89b211c84c4228b9ae" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 40381 } }
I20260809 02:02:29.594445 3538 ts_tablet_manager.cc:1428] T 76aaf83ff0a94d3c9994c8eda3457f17 P 374d1f9d096c4e89b211c84c4228b9ae: Time spent starting tablet: real 0.006s user 0.000s sys 0.006s
I20260809 02:02:29.594904 3364 heartbeater.cc:503] Master 127.1.192.254:39235 was elected leader, sending a full tablet report...
I20260809 02:02:29.595966 3536 raft_consensus.cc:348] T 76aaf83ff0a94d3c9994c8eda3457f17 P 905d1a29a3c4431499499b3df47ecaf4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "350e9b1bbad84279bf35b4e4db434b6b" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 40369 } } peers { permanent_uuid: "905d1a29a3c4431499499b3df47ecaf4" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 42657 } } peers { permanent_uuid: "374d1f9d096c4e89b211c84c4228b9ae" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 40381 } }
I20260809 02:02:29.596282 3536 raft_consensus.cc:374] T 76aaf83ff0a94d3c9994c8eda3457f17 P 905d1a29a3c4431499499b3df47ecaf4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:02:29.596349 3536 raft_consensus.cc:729] T 76aaf83ff0a94d3c9994c8eda3457f17 P 905d1a29a3c4431499499b3df47ecaf4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 905d1a29a3c4431499499b3df47ecaf4, State: Initialized, Role: FOLLOWER
I20260809 02:02:29.596827 3536 consensus_queue.cc:260] T 76aaf83ff0a94d3c9994c8eda3457f17 P 905d1a29a3c4431499499b3df47ecaf4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "350e9b1bbad84279bf35b4e4db434b6b" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 40369 } } peers { permanent_uuid: "905d1a29a3c4431499499b3df47ecaf4" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 42657 } } peers { permanent_uuid: "374d1f9d096c4e89b211c84c4228b9ae" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 40381 } }
I20260809 02:02:29.596979 3535 raft_consensus.cc:348] T 76aaf83ff0a94d3c9994c8eda3457f17 P 350e9b1bbad84279bf35b4e4db434b6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "350e9b1bbad84279bf35b4e4db434b6b" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 40369 } } peers { permanent_uuid: "905d1a29a3c4431499499b3df47ecaf4" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 42657 } } peers { permanent_uuid: "374d1f9d096c4e89b211c84c4228b9ae" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 40381 } }
I20260809 02:02:29.597288 3535 raft_consensus.cc:374] T 76aaf83ff0a94d3c9994c8eda3457f17 P 350e9b1bbad84279bf35b4e4db434b6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:02:29.597366 3535 raft_consensus.cc:729] T 76aaf83ff0a94d3c9994c8eda3457f17 P 350e9b1bbad84279bf35b4e4db434b6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 350e9b1bbad84279bf35b4e4db434b6b, State: Initialized, Role: FOLLOWER
I20260809 02:02:29.597796 3509 heartbeater.cc:503] Master 127.1.192.254:39235 was elected leader, sending a full tablet report...
I20260809 02:02:29.597867 3535 consensus_queue.cc:260] T 76aaf83ff0a94d3c9994c8eda3457f17 P 350e9b1bbad84279bf35b4e4db434b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "350e9b1bbad84279bf35b4e4db434b6b" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 40369 } } peers { permanent_uuid: "905d1a29a3c4431499499b3df47ecaf4" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 42657 } } peers { permanent_uuid: "374d1f9d096c4e89b211c84c4228b9ae" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 40381 } }
I20260809 02:02:29.598040 3536 ts_tablet_manager.cc:1428] T 76aaf83ff0a94d3c9994c8eda3457f17 P 905d1a29a3c4431499499b3df47ecaf4: Time spent starting tablet: real 0.006s user 0.002s sys 0.005s
I20260809 02:02:29.598948 3535 ts_tablet_manager.cc:1428] T 76aaf83ff0a94d3c9994c8eda3457f17 P 350e9b1bbad84279bf35b4e4db434b6b: Time spent starting tablet: real 0.007s user 0.007s sys 0.001s
I20260809 02:02:29.599115 3208 heartbeater.cc:503] Master 127.1.192.254:39235 was elected leader, sending a full tablet report...
I20260809 02:02:29.714874 3543 raft_consensus.cc:482] T 76aaf83ff0a94d3c9994c8eda3457f17 P 905d1a29a3c4431499499b3df47ecaf4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:02:29.715207 3543 raft_consensus.cc:504] T 76aaf83ff0a94d3c9994c8eda3457f17 P 905d1a29a3c4431499499b3df47ecaf4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "350e9b1bbad84279bf35b4e4db434b6b" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 40369 } } peers { permanent_uuid: "905d1a29a3c4431499499b3df47ecaf4" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 42657 } } peers { permanent_uuid: "374d1f9d096c4e89b211c84c4228b9ae" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 40381 } }
I20260809 02:02:29.716662 3543 leader_election.cc:288] T 76aaf83ff0a94d3c9994c8eda3457f17 P 905d1a29a3c4431499499b3df47ecaf4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 350e9b1bbad84279bf35b4e4db434b6b (127.1.192.193:40369), 374d1f9d096c4e89b211c84c4228b9ae (127.1.192.194:40381)
I20260809 02:02:29.721971 3399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:29.717035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.194:40381 (local address 127.1.192.195:58885)
0809 02:02:29.717166 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:29.717181 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 02:02:29.717290 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:02:29.717721 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:02:29.717731 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:02:29.717748 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:02:29.717844 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:29.717849 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:29.719327 (+ 1478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:29.719331 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:29.720623 (+ 1292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:29.720634 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:29.721600 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:29.721603 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:29.721710 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:29.721730 (+ 20us) client_negotiation.cc:769] Sending connection context
0809 02:02:29.721793 (+ 63us) client_negotiation.cc:240] Negotiation successful
0809 02:02:29.721856 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260809 02:02:29.722100 3533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:29.717179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.195:58885 (local address 127.1.192.194:40381)
0809 02:02:29.717334 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:29.717338 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:29.717353 (+ 15us) server_negotiation.cc:407] Connection header received
0809 02:02:29.717401 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:29.717403 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:29.717443 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:29.717561 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:02:29.718240 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:29.719186 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:29.720764 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:29.721408 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:29.721596 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:29.721657 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:29.721862 (+ 205us) server_negotiation.cc:299] Negotiation successful
0809 02:02:29.721957 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 02:02:29.722779 3313 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "76aaf83ff0a94d3c9994c8eda3457f17" candidate_uuid: "905d1a29a3c4431499499b3df47ecaf4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "374d1f9d096c4e89b211c84c4228b9ae" is_pre_election: true
I20260809 02:02:29.723120 3313 raft_consensus.cc:2454] T 76aaf83ff0a94d3c9994c8eda3457f17 P 374d1f9d096c4e89b211c84c4228b9ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 905d1a29a3c4431499499b3df47ecaf4 in term 0.
I20260809 02:02:29.723711 3391 leader_election.cc:302] T 76aaf83ff0a94d3c9994c8eda3457f17 P 905d1a29a3c4431499499b3df47ecaf4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 374d1f9d096c4e89b211c84c4228b9ae, 905d1a29a3c4431499499b3df47ecaf4; no voters:
I20260809 02:02:29.723917 3545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:29.717044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.193:40369 (local address 127.1.192.195:39055)
0809 02:02:29.717793 (+ 749us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:29.717804 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 02:02:29.717892 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:02:29.718187 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:02:29.718192 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:02:29.718213 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:02:29.718348 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:29.718355 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:29.719351 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:29.719354 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:29.721387 (+ 2033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:29.721403 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:29.723449 (+ 2046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:29.723452 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:29.723579 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:29.723603 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 02:02:29.723685 (+ 82us) client_negotiation.cc:240] Negotiation successful
0809 02:02:29.723765 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":685,"mutex_wait_us":323,"thread_start_us":255,"threads_started":1}
I20260809 02:02:29.723939 3528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:29.717189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.195:39055 (local address 127.1.192.193:40369)
0809 02:02:29.717360 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:29.717366 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:29.717891 (+ 525us) server_negotiation.cc:407] Connection header received
0809 02:02:29.717999 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:29.718003 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:29.718060 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:29.718156 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:02:29.718490 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:29.719219 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:29.721644 (+ 2425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:29.722405 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:29.722615 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:29.722664 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:29.723743 (+ 1079us) server_negotiation.cc:299] Negotiation successful
0809 02:02:29.723829 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 02:02:29.724051 3543 raft_consensus.cc:2781] T 76aaf83ff0a94d3c9994c8eda3457f17 P 905d1a29a3c4431499499b3df47ecaf4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:02:29.724145 3543 raft_consensus.cc:482] T 76aaf83ff0a94d3c9994c8eda3457f17 P 905d1a29a3c4431499499b3df47ecaf4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:02:29.724211 3543 raft_consensus.cc:3037] T 76aaf83ff0a94d3c9994c8eda3457f17 P 905d1a29a3c4431499499b3df47ecaf4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:02:29.724725 3157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "76aaf83ff0a94d3c9994c8eda3457f17" candidate_uuid: "905d1a29a3c4431499499b3df47ecaf4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "350e9b1bbad84279bf35b4e4db434b6b" is_pre_election: true
I20260809 02:02:29.725054 3157 raft_consensus.cc:2454] T 76aaf83ff0a94d3c9994c8eda3457f17 P 350e9b1bbad84279bf35b4e4db434b6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 905d1a29a3c4431499499b3df47ecaf4 in term 0.
I20260809 02:02:29.725591 3543 raft_consensus.cc:504] T 76aaf83ff0a94d3c9994c8eda3457f17 P 905d1a29a3c4431499499b3df47ecaf4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "350e9b1bbad84279bf35b4e4db434b6b" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 40369 } } peers { permanent_uuid: "905d1a29a3c4431499499b3df47ecaf4" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 42657 } } peers { permanent_uuid: "374d1f9d096c4e89b211c84c4228b9ae" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 40381 } }
I20260809 02:02:29.725973 3543 leader_election.cc:288] T 76aaf83ff0a94d3c9994c8eda3457f17 P 905d1a29a3c4431499499b3df47ecaf4 [CANDIDATE]: Term 1 election: Requested vote from peers 350e9b1bbad84279bf35b4e4db434b6b (127.1.192.193:40369), 374d1f9d096c4e89b211c84c4228b9ae (127.1.192.194:40381)
I20260809 02:02:29.726361 3313 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "76aaf83ff0a94d3c9994c8eda3457f17" candidate_uuid: "905d1a29a3c4431499499b3df47ecaf4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "374d1f9d096c4e89b211c84c4228b9ae"
I20260809 02:02:29.726485 3313 raft_consensus.cc:3037] T 76aaf83ff0a94d3c9994c8eda3457f17 P 374d1f9d096c4e89b211c84c4228b9ae [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:02:29.726528 3157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "76aaf83ff0a94d3c9994c8eda3457f17" candidate_uuid: "905d1a29a3c4431499499b3df47ecaf4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "350e9b1bbad84279bf35b4e4db434b6b"
I20260809 02:02:29.726653 3157 raft_consensus.cc:3037] T 76aaf83ff0a94d3c9994c8eda3457f17 P 350e9b1bbad84279bf35b4e4db434b6b [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:02:29.727690 3157 raft_consensus.cc:2454] T 76aaf83ff0a94d3c9994c8eda3457f17 P 350e9b1bbad84279bf35b4e4db434b6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 905d1a29a3c4431499499b3df47ecaf4 in term 1.
I20260809 02:02:29.727701 3313 raft_consensus.cc:2454] T 76aaf83ff0a94d3c9994c8eda3457f17 P 374d1f9d096c4e89b211c84c4228b9ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 905d1a29a3c4431499499b3df47ecaf4 in term 1.
I20260809 02:02:29.728073 3394 leader_election.cc:302] T 76aaf83ff0a94d3c9994c8eda3457f17 P 905d1a29a3c4431499499b3df47ecaf4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 350e9b1bbad84279bf35b4e4db434b6b, 905d1a29a3c4431499499b3df47ecaf4; no voters:
I20260809 02:02:29.728298 3543 raft_consensus.cc:2781] T 76aaf83ff0a94d3c9994c8eda3457f17 P 905d1a29a3c4431499499b3df47ecaf4 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:02:29.728575 3543 raft_consensus.cc:686] T 76aaf83ff0a94d3c9994c8eda3457f17 P 905d1a29a3c4431499499b3df47ecaf4 [term 1 LEADER]: Becoming Leader. State: Replica: 905d1a29a3c4431499499b3df47ecaf4, State: Running, Role: LEADER
I20260809 02:02:29.728956 3543 consensus_queue.cc:237] T 76aaf83ff0a94d3c9994c8eda3457f17 P 905d1a29a3c4431499499b3df47ecaf4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "350e9b1bbad84279bf35b4e4db434b6b" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 40369 } } peers { permanent_uuid: "905d1a29a3c4431499499b3df47ecaf4" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 42657 } } peers { permanent_uuid: "374d1f9d096c4e89b211c84c4228b9ae" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 40381 } }
I20260809 02:02:29.732903 2991 catalog_manager.cc:5495] T 76aaf83ff0a94d3c9994c8eda3457f17 P 905d1a29a3c4431499499b3df47ecaf4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 905d1a29a3c4431499499b3df47ecaf4 (127.1.192.195). New cstate: current_term: 1 leader_uuid: "905d1a29a3c4431499499b3df47ecaf4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "350e9b1bbad84279bf35b4e4db434b6b" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 40369 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "905d1a29a3c4431499499b3df47ecaf4" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 42657 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "374d1f9d096c4e89b211c84c4228b9ae" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 40381 } health_report { overall_health: UNKNOWN } } }
W20260809 02:02:29.738858 3365 tablet.cc:2369] T 76aaf83ff0a94d3c9994c8eda3457f17 P 374d1f9d096c4e89b211c84c4228b9ae: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 02:02:29.745756 3209 tablet.cc:2369] T 76aaf83ff0a94d3c9994c8eda3457f17 P 350e9b1bbad84279bf35b4e4db434b6b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 02:02:29.752839 3510 tablet.cc:2369] T 76aaf83ff0a94d3c9994c8eda3457f17 P 905d1a29a3c4431499499b3df47ecaf4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 02:02:29.769927 3534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:29.765972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60732 (local address 127.1.192.195:42657)
0809 02:02:29.766103 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:29.766106 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:29.766143 (+ 37us) server_negotiation.cc:407] Connection header received
0809 02:02:29.766386 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:29.766388 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:29.766445 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:29.766539 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:02:29.766785 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:29.767291 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:29.768245 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:29.768469 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:29.768555 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:29.768941 (+ 386us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:02:29.769429 (+ 488us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:02:29.769659 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:29.769752 (+ 93us) server_negotiation.cc:299] Negotiation successful
0809 02:02:29.769809 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 02:02:29.776767 3157 raft_consensus.cc:1261] T 76aaf83ff0a94d3c9994c8eda3457f17 P 350e9b1bbad84279bf35b4e4db434b6b [term 1 FOLLOWER]: Refusing update from remote peer 905d1a29a3c4431499499b3df47ecaf4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:02:29.776767 3313 raft_consensus.cc:1261] T 76aaf83ff0a94d3c9994c8eda3457f17 P 374d1f9d096c4e89b211c84c4228b9ae [term 1 FOLLOWER]: Refusing update from remote peer 905d1a29a3c4431499499b3df47ecaf4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:02:29.777629 3543 consensus_queue.cc:1035] T 76aaf83ff0a94d3c9994c8eda3457f17 P 905d1a29a3c4431499499b3df47ecaf4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "350e9b1bbad84279bf35b4e4db434b6b" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 40369 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:02:29.777879 3546 consensus_queue.cc:1035] T 76aaf83ff0a94d3c9994c8eda3457f17 P 905d1a29a3c4431499499b3df47ecaf4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "374d1f9d096c4e89b211c84c4228b9ae" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 40381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:02:29.786078 3550 mvcc.cc:205] Tried to move back new op lower bound from 7316442930276982784 to 7316442930094972928. Current Snapshot: MvccSnapshot[applied={T|T < 7316442930276982784}]
I20260809 02:02:29.786475 3551 mvcc.cc:205] Tried to move back new op lower bound from 7316442930276982784 to 7316442930094972928. Current Snapshot: MvccSnapshot[applied={T|T < 7316442930276982784}]
I20260809 02:02:29.800320 2992 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43800:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:02:29.800617 2992 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43800:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:02:29.803651 2992 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 484e26ab9179442793b9906de0294663: Sending DeleteTablet for 3 replicas of tablet 76aaf83ff0a94d3c9994c8eda3457f17
I20260809 02:02:29.804737 3440 tablet_service.cc:1514] Processing DeleteTablet for tablet 76aaf83ff0a94d3c9994c8eda3457f17 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:02:29 UTC) from {username='kudu', principal='kudu/127.1.192.254@KRBTEST.COM'} at 127.0.0.1:60718
I20260809 02:02:29.804738 3136 tablet_service.cc:1514] Processing DeleteTablet for tablet 76aaf83ff0a94d3c9994c8eda3457f17 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:02:29 UTC) from {username='kudu', principal='kudu/127.1.192.254@KRBTEST.COM'} at 127.0.0.1:45388
I20260809 02:02:29.805433 3291 tablet_service.cc:1514] Processing DeleteTablet for tablet 76aaf83ff0a94d3c9994c8eda3457f17 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:02:29 UTC) from {username='kudu', principal='kudu/127.1.192.254@KRBTEST.COM'} at 127.0.0.1:45016
I20260809 02:02:29.805727 2992 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43800:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 02:02:29.806104 2992 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:02:29.806344 3560 tablet_replica.cc:330] T 76aaf83ff0a94d3c9994c8eda3457f17 P 350e9b1bbad84279bf35b4e4db434b6b: stopping tablet replica
I20260809 02:02:29.806794 3560 raft_consensus.cc:2229] T 76aaf83ff0a94d3c9994c8eda3457f17 P 350e9b1bbad84279bf35b4e4db434b6b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:02:29.807217 3561 tablet_replica.cc:330] T 76aaf83ff0a94d3c9994c8eda3457f17 P 374d1f9d096c4e89b211c84c4228b9ae: stopping tablet replica
I20260809 02:02:29.807449 3561 raft_consensus.cc:2229] T 76aaf83ff0a94d3c9994c8eda3457f17 P 374d1f9d096c4e89b211c84c4228b9ae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:02:29.807777 3561 raft_consensus.cc:2258] T 76aaf83ff0a94d3c9994c8eda3457f17 P 374d1f9d096c4e89b211c84c4228b9ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:02:29.808495 3560 raft_consensus.cc:2258] T 76aaf83ff0a94d3c9994c8eda3457f17 P 350e9b1bbad84279bf35b4e4db434b6b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:02:29.808902 3559 tablet_replica.cc:330] T 76aaf83ff0a94d3c9994c8eda3457f17 P 905d1a29a3c4431499499b3df47ecaf4: stopping tablet replica
I20260809 02:02:29.809213 3559 raft_consensus.cc:2229] T 76aaf83ff0a94d3c9994c8eda3457f17 P 905d1a29a3c4431499499b3df47ecaf4 [term 1 LEADER]: Raft consensus shutting down.
I20260809 02:02:29.809558 3561 ts_tablet_manager.cc:1905] T 76aaf83ff0a94d3c9994c8eda3457f17 P 374d1f9d096c4e89b211c84c4228b9ae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:02:29.809656 3559 raft_consensus.cc:2258] T 76aaf83ff0a94d3c9994c8eda3457f17 P 905d1a29a3c4431499499b3df47ecaf4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:02:29.811472 3560 ts_tablet_manager.cc:1905] T 76aaf83ff0a94d3c9994c8eda3457f17 P 350e9b1bbad84279bf35b4e4db434b6b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:02:29.811973 3136 tablet_service.cc:1467] Processing CreateTablet for tablet 1c670ba19f304e979a9405aadd4c0aca (DEFAULT_TABLE table=test-table [id=12fa81fec6a946979a23a9dcf25054a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:02:29.812106 3440 tablet_service.cc:1467] Processing CreateTablet for tablet 1c670ba19f304e979a9405aadd4c0aca (DEFAULT_TABLE table=test-table [id=12fa81fec6a946979a23a9dcf25054a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:02:29.812252 3136 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c670ba19f304e979a9405aadd4c0aca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:02:29.812379 3440 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c670ba19f304e979a9405aadd4c0aca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:02:29.814708 3560 ts_tablet_manager.cc:1918] T 76aaf83ff0a94d3c9994c8eda3457f17 P 350e9b1bbad84279bf35b4e4db434b6b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:02:29.815596 3291 tablet_service.cc:1467] Processing CreateTablet for tablet 1c670ba19f304e979a9405aadd4c0aca (DEFAULT_TABLE table=test-table [id=12fa81fec6a946979a23a9dcf25054a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:02:29.815845 3560 log.cc:1196] T 76aaf83ff0a94d3c9994c8eda3457f17 P 350e9b1bbad84279bf35b4e4db434b6b: Deleting WAL directory at /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786240941968412-1795-0/minicluster-data/ts-0/wal/wals/76aaf83ff0a94d3c9994c8eda3457f17
I20260809 02:02:29.815877 3291 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c670ba19f304e979a9405aadd4c0aca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:02:29.816404 3560 ts_tablet_manager.cc:1939] T 76aaf83ff0a94d3c9994c8eda3457f17 P 350e9b1bbad84279bf35b4e4db434b6b: Deleting consensus metadata
I20260809 02:02:29.816740 3559 ts_tablet_manager.cc:1905] T 76aaf83ff0a94d3c9994c8eda3457f17 P 905d1a29a3c4431499499b3df47ecaf4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:02:29.816893 3536 tablet_bootstrap.cc:492] T 1c670ba19f304e979a9405aadd4c0aca P 905d1a29a3c4431499499b3df47ecaf4: Bootstrap starting.
I20260809 02:02:29.818249 3538 tablet_bootstrap.cc:492] T 1c670ba19f304e979a9405aadd4c0aca P 374d1f9d096c4e89b211c84c4228b9ae: Bootstrap starting.
I20260809 02:02:29.814906 3535 tablet_bootstrap.cc:492] T 1c670ba19f304e979a9405aadd4c0aca P 350e9b1bbad84279bf35b4e4db434b6b: Bootstrap starting.
I20260809 02:02:29.818596 3536 tablet_bootstrap.cc:654] T 1c670ba19f304e979a9405aadd4c0aca P 905d1a29a3c4431499499b3df47ecaf4: Neither blocks nor log segments found. Creating new log.
I20260809 02:02:29.819248 3538 tablet_bootstrap.cc:654] T 1c670ba19f304e979a9405aadd4c0aca P 374d1f9d096c4e89b211c84c4228b9ae: Neither blocks nor log segments found. Creating new log.
I20260809 02:02:29.819293 3535 tablet_bootstrap.cc:654] T 1c670ba19f304e979a9405aadd4c0aca P 350e9b1bbad84279bf35b4e4db434b6b: Neither blocks nor log segments found. Creating new log.
I20260809 02:02:29.821008 2979 catalog_manager.cc:4841] TS 350e9b1bbad84279bf35b4e4db434b6b (127.1.192.193:40369): tablet 76aaf83ff0a94d3c9994c8eda3457f17 (table test-table [id=4c7442b6c12341888d847bfab0b57582]) successfully deleted
I20260809 02:02:29.821444 3559 ts_tablet_manager.cc:1918] T 76aaf83ff0a94d3c9994c8eda3457f17 P 905d1a29a3c4431499499b3df47ecaf4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:02:29.821595 3559 log.cc:1196] T 76aaf83ff0a94d3c9994c8eda3457f17 P 905d1a29a3c4431499499b3df47ecaf4: Deleting WAL directory at /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786240941968412-1795-0/minicluster-data/ts-2/wal/wals/76aaf83ff0a94d3c9994c8eda3457f17
I20260809 02:02:29.821728 3535 tablet_bootstrap.cc:492] T 1c670ba19f304e979a9405aadd4c0aca P 350e9b1bbad84279bf35b4e4db434b6b: No bootstrap required, opened a new log
I20260809 02:02:29.821830 3535 ts_tablet_manager.cc:1397] T 1c670ba19f304e979a9405aadd4c0aca P 350e9b1bbad84279bf35b4e4db434b6b: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.001s
I20260809 02:02:29.822043 3559 ts_tablet_manager.cc:1939] T 76aaf83ff0a94d3c9994c8eda3457f17 P 905d1a29a3c4431499499b3df47ecaf4: Deleting consensus metadata
I20260809 02:02:29.822378 3535 raft_consensus.cc:348] T 1c670ba19f304e979a9405aadd4c0aca P 350e9b1bbad84279bf35b4e4db434b6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "350e9b1bbad84279bf35b4e4db434b6b" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 40369 } } peers { permanent_uuid: "374d1f9d096c4e89b211c84c4228b9ae" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 40381 } } peers { permanent_uuid: "905d1a29a3c4431499499b3df47ecaf4" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 42657 } }
I20260809 02:02:29.822508 3535 raft_consensus.cc:374] T 1c670ba19f304e979a9405aadd4c0aca P 350e9b1bbad84279bf35b4e4db434b6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:02:29.822542 3535 raft_consensus.cc:729] T 1c670ba19f304e979a9405aadd4c0aca P 350e9b1bbad84279bf35b4e4db434b6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 350e9b1bbad84279bf35b4e4db434b6b, State: Initialized, Role: FOLLOWER
I20260809 02:02:29.822701 3535 consensus_queue.cc:260] T 1c670ba19f304e979a9405aadd4c0aca P 350e9b1bbad84279bf35b4e4db434b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "350e9b1bbad84279bf35b4e4db434b6b" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 40369 } } peers { permanent_uuid: "374d1f9d096c4e89b211c84c4228b9ae" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 40381 } } peers { permanent_uuid: "905d1a29a3c4431499499b3df47ecaf4" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 42657 } }
I20260809 02:02:29.823575 2979 catalog_manager.cc:4841] TS 905d1a29a3c4431499499b3df47ecaf4 (127.1.192.195:42657): tablet 76aaf83ff0a94d3c9994c8eda3457f17 (table test-table [id=4c7442b6c12341888d847bfab0b57582]) successfully deleted
I20260809 02:02:29.823876 3535 ts_tablet_manager.cc:1428] T 1c670ba19f304e979a9405aadd4c0aca P 350e9b1bbad84279bf35b4e4db434b6b: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260809 02:02:29.825521 3561 ts_tablet_manager.cc:1918] T 76aaf83ff0a94d3c9994c8eda3457f17 P 374d1f9d096c4e89b211c84c4228b9ae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:02:29.825661 3561 log.cc:1196] T 76aaf83ff0a94d3c9994c8eda3457f17 P 374d1f9d096c4e89b211c84c4228b9ae: Deleting WAL directory at /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786240941968412-1795-0/minicluster-data/ts-1/wal/wals/76aaf83ff0a94d3c9994c8eda3457f17
I20260809 02:02:29.826125 3561 ts_tablet_manager.cc:1939] T 76aaf83ff0a94d3c9994c8eda3457f17 P 374d1f9d096c4e89b211c84c4228b9ae: Deleting consensus metadata
I20260809 02:02:29.827131 3536 tablet_bootstrap.cc:492] T 1c670ba19f304e979a9405aadd4c0aca P 905d1a29a3c4431499499b3df47ecaf4: No bootstrap required, opened a new log
I20260809 02:02:29.827235 3536 ts_tablet_manager.cc:1397] T 1c670ba19f304e979a9405aadd4c0aca P 905d1a29a3c4431499499b3df47ecaf4: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20260809 02:02:29.827677 2975 catalog_manager.cc:4841] TS 374d1f9d096c4e89b211c84c4228b9ae (127.1.192.194:40381): tablet 76aaf83ff0a94d3c9994c8eda3457f17 (table test-table [id=4c7442b6c12341888d847bfab0b57582]) successfully deleted
I20260809 02:02:29.827757 3536 raft_consensus.cc:348] T 1c670ba19f304e979a9405aadd4c0aca P 905d1a29a3c4431499499b3df47ecaf4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "350e9b1bbad84279bf35b4e4db434b6b" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 40369 } } peers { permanent_uuid: "374d1f9d096c4e89b211c84c4228b9ae" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 40381 } } peers { permanent_uuid: "905d1a29a3c4431499499b3df47ecaf4" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 42657 } }
I20260809 02:02:29.827910 3536 raft_consensus.cc:374] T 1c670ba19f304e979a9405aadd4c0aca P 905d1a29a3c4431499499b3df47ecaf4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:02:29.827947 3536 raft_consensus.cc:729] T 1c670ba19f304e979a9405aadd4c0aca P 905d1a29a3c4431499499b3df47ecaf4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 905d1a29a3c4431499499b3df47ecaf4, State: Initialized, Role: FOLLOWER
I20260809 02:02:29.828033 3538 tablet_bootstrap.cc:492] T 1c670ba19f304e979a9405aadd4c0aca P 374d1f9d096c4e89b211c84c4228b9ae: No bootstrap required, opened a new log
I20260809 02:02:29.828150 3538 ts_tablet_manager.cc:1397] T 1c670ba19f304e979a9405aadd4c0aca P 374d1f9d096c4e89b211c84c4228b9ae: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.002s
I20260809 02:02:29.828132 3536 consensus_queue.cc:260] T 1c670ba19f304e979a9405aadd4c0aca P 905d1a29a3c4431499499b3df47ecaf4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "350e9b1bbad84279bf35b4e4db434b6b" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 40369 } } peers { permanent_uuid: "374d1f9d096c4e89b211c84c4228b9ae" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 40381 } } peers { permanent_uuid: "905d1a29a3c4431499499b3df47ecaf4" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 42657 } }
I20260809 02:02:29.828438 3536 ts_tablet_manager.cc:1428] T 1c670ba19f304e979a9405aadd4c0aca P 905d1a29a3c4431499499b3df47ecaf4: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260809 02:02:29.828650 3538 raft_consensus.cc:348] T 1c670ba19f304e979a9405aadd4c0aca P 374d1f9d096c4e89b211c84c4228b9ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "350e9b1bbad84279bf35b4e4db434b6b" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 40369 } } peers { permanent_uuid: "374d1f9d096c4e89b211c84c4228b9ae" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 40381 } } peers { permanent_uuid: "905d1a29a3c4431499499b3df47ecaf4" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 42657 } }
I20260809 02:02:29.828768 3538 raft_consensus.cc:374] T 1c670ba19f304e979a9405aadd4c0aca P 374d1f9d096c4e89b211c84c4228b9ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:02:29.828809 3538 raft_consensus.cc:729] T 1c670ba19f304e979a9405aadd4c0aca P 374d1f9d096c4e89b211c84c4228b9ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 374d1f9d096c4e89b211c84c4228b9ae, State: Initialized, Role: FOLLOWER
I20260809 02:02:29.828951 3538 consensus_queue.cc:260] T 1c670ba19f304e979a9405aadd4c0aca P 374d1f9d096c4e89b211c84c4228b9ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "350e9b1bbad84279bf35b4e4db434b6b" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 40369 } } peers { permanent_uuid: "374d1f9d096c4e89b211c84c4228b9ae" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 40381 } } peers { permanent_uuid: "905d1a29a3c4431499499b3df47ecaf4" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 42657 } }
I20260809 02:02:29.829228 3538 ts_tablet_manager.cc:1428] T 1c670ba19f304e979a9405aadd4c0aca P 374d1f9d096c4e89b211c84c4228b9ae: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260809 02:02:30.047799 3542 raft_consensus.cc:482] T 1c670ba19f304e979a9405aadd4c0aca P 374d1f9d096c4e89b211c84c4228b9ae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:02:30.047986 3542 raft_consensus.cc:504] T 1c670ba19f304e979a9405aadd4c0aca P 374d1f9d096c4e89b211c84c4228b9ae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "350e9b1bbad84279bf35b4e4db434b6b" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 40369 } } peers { permanent_uuid: "374d1f9d096c4e89b211c84c4228b9ae" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 40381 } } peers { permanent_uuid: "905d1a29a3c4431499499b3df47ecaf4" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 42657 } }
I20260809 02:02:30.049093 3542 leader_election.cc:288] T 1c670ba19f304e979a9405aadd4c0aca P 374d1f9d096c4e89b211c84c4228b9ae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 350e9b1bbad84279bf35b4e4db434b6b (127.1.192.193:40369), 905d1a29a3c4431499499b3df47ecaf4 (127.1.192.195:42657)
I20260809 02:02:30.053819 3565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:30.049536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.193:40369 (local address 127.1.192.194:51351)
0809 02:02:30.049835 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:30.049855 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 02:02:30.050054 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:02:30.050517 (+ 463us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:02:30.050520 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:02:30.050539 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:02:30.050662 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:30.050669 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:30.051592 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:30.051596 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:30.052515 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:30.052524 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:30.053324 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:30.053331 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:30.053430 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:30.053460 (+ 30us) client_negotiation.cc:769] Sending connection context
0809 02:02:30.053555 (+ 95us) client_negotiation.cc:240] Negotiation successful
0809 02:02:30.053653 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":87,"threads_started":1}
I20260809 02:02:30.053838 3528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:30.049626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.194:51351 (local address 127.1.192.193:40369)
0809 02:02:30.049791 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:30.049797 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:30.049965 (+ 168us) server_negotiation.cc:407] Connection header received
0809 02:02:30.050233 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:30.050238 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:30.050285 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:30.050335 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:02:30.050795 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:30.051472 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:30.052657 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:30.053170 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:30.053273 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:30.053352 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:30.053585 (+ 233us) server_negotiation.cc:299] Negotiation successful
0809 02:02:30.053691 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 02:02:30.053894 3534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:30.050070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.194:36639 (local address 127.1.192.195:42657)
0809 02:02:30.050211 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:30.050215 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:30.050414 (+ 199us) server_negotiation.cc:407] Connection header received
0809 02:02:30.050634 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:30.050638 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:30.050693 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:30.050795 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:02:30.051100 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:30.051682 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:30.052819 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:30.053249 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:30.053373 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:30.053428 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:30.053696 (+ 268us) server_negotiation.cc:299] Negotiation successful
0809 02:02:30.053773 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 02:02:30.053874 3566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:30.049936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.195:42657 (local address 127.1.192.194:36639)
0809 02:02:30.050308 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:30.050323 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 02:02:30.050436 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:02:30.050809 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:02:30.050812 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:02:30.050829 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:02:30.050961 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:30.050968 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:30.051791 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:30.051793 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:30.052689 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:30.052699 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:30.053389 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:30.053393 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:30.053549 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:30.053576 (+ 27us) client_negotiation.cc:769] Sending connection context
0809 02:02:30.053660 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 02:02:30.053727 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":102,"threads_started":1}
I20260809 02:02:30.054294 3157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c670ba19f304e979a9405aadd4c0aca" candidate_uuid: "374d1f9d096c4e89b211c84c4228b9ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "350e9b1bbad84279bf35b4e4db434b6b" is_pre_election: true
I20260809 02:02:30.054474 3157 raft_consensus.cc:2454] T 1c670ba19f304e979a9405aadd4c0aca P 350e9b1bbad84279bf35b4e4db434b6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 374d1f9d096c4e89b211c84c4228b9ae in term 0.
I20260809 02:02:30.054564 3460 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c670ba19f304e979a9405aadd4c0aca" candidate_uuid: "374d1f9d096c4e89b211c84c4228b9ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "905d1a29a3c4431499499b3df47ecaf4" is_pre_election: true
I20260809 02:02:30.054798 3460 raft_consensus.cc:2454] T 1c670ba19f304e979a9405aadd4c0aca P 905d1a29a3c4431499499b3df47ecaf4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 374d1f9d096c4e89b211c84c4228b9ae in term 0.
I20260809 02:02:30.054860 3240 leader_election.cc:302] T 1c670ba19f304e979a9405aadd4c0aca P 374d1f9d096c4e89b211c84c4228b9ae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 350e9b1bbad84279bf35b4e4db434b6b, 374d1f9d096c4e89b211c84c4228b9ae; no voters:
I20260809 02:02:30.055203 3542 raft_consensus.cc:2781] T 1c670ba19f304e979a9405aadd4c0aca P 374d1f9d096c4e89b211c84c4228b9ae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:02:30.055289 3542 raft_consensus.cc:482] T 1c670ba19f304e979a9405aadd4c0aca P 374d1f9d096c4e89b211c84c4228b9ae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:02:30.055321 3542 raft_consensus.cc:3037] T 1c670ba19f304e979a9405aadd4c0aca P 374d1f9d096c4e89b211c84c4228b9ae [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:02:30.056483 3542 raft_consensus.cc:504] T 1c670ba19f304e979a9405aadd4c0aca P 374d1f9d096c4e89b211c84c4228b9ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "350e9b1bbad84279bf35b4e4db434b6b" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 40369 } } peers { permanent_uuid: "374d1f9d096c4e89b211c84c4228b9ae" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 40381 } } peers { permanent_uuid: "905d1a29a3c4431499499b3df47ecaf4" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 42657 } }
I20260809 02:02:30.056870 3542 leader_election.cc:288] T 1c670ba19f304e979a9405aadd4c0aca P 374d1f9d096c4e89b211c84c4228b9ae [CANDIDATE]: Term 1 election: Requested vote from peers 350e9b1bbad84279bf35b4e4db434b6b (127.1.192.193:40369), 905d1a29a3c4431499499b3df47ecaf4 (127.1.192.195:42657)
I20260809 02:02:30.057303 3157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c670ba19f304e979a9405aadd4c0aca" candidate_uuid: "374d1f9d096c4e89b211c84c4228b9ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "350e9b1bbad84279bf35b4e4db434b6b"
I20260809 02:02:30.057372 3460 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c670ba19f304e979a9405aadd4c0aca" candidate_uuid: "374d1f9d096c4e89b211c84c4228b9ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "905d1a29a3c4431499499b3df47ecaf4"
I20260809 02:02:30.057451 3157 raft_consensus.cc:3037] T 1c670ba19f304e979a9405aadd4c0aca P 350e9b1bbad84279bf35b4e4db434b6b [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:02:30.057534 3460 raft_consensus.cc:3037] T 1c670ba19f304e979a9405aadd4c0aca P 905d1a29a3c4431499499b3df47ecaf4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:02:30.058553 3157 raft_consensus.cc:2454] T 1c670ba19f304e979a9405aadd4c0aca P 350e9b1bbad84279bf35b4e4db434b6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 374d1f9d096c4e89b211c84c4228b9ae in term 1.
I20260809 02:02:30.058725 3460 raft_consensus.cc:2454] T 1c670ba19f304e979a9405aadd4c0aca P 905d1a29a3c4431499499b3df47ecaf4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 374d1f9d096c4e89b211c84c4228b9ae in term 1.
I20260809 02:02:30.058887 3240 leader_election.cc:302] T 1c670ba19f304e979a9405aadd4c0aca P 374d1f9d096c4e89b211c84c4228b9ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 350e9b1bbad84279bf35b4e4db434b6b, 374d1f9d096c4e89b211c84c4228b9ae; no voters:
I20260809 02:02:30.059106 3542 raft_consensus.cc:2781] T 1c670ba19f304e979a9405aadd4c0aca P 374d1f9d096c4e89b211c84c4228b9ae [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:02:30.059350 3542 raft_consensus.cc:686] T 1c670ba19f304e979a9405aadd4c0aca P 374d1f9d096c4e89b211c84c4228b9ae [term 1 LEADER]: Becoming Leader. State: Replica: 374d1f9d096c4e89b211c84c4228b9ae, State: Running, Role: LEADER
I20260809 02:02:30.059616 3542 consensus_queue.cc:237] T 1c670ba19f304e979a9405aadd4c0aca P 374d1f9d096c4e89b211c84c4228b9ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "350e9b1bbad84279bf35b4e4db434b6b" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 40369 } } peers { permanent_uuid: "374d1f9d096c4e89b211c84c4228b9ae" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 40381 } } peers { permanent_uuid: "905d1a29a3c4431499499b3df47ecaf4" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 42657 } }
I20260809 02:02:30.062095 2992 catalog_manager.cc:5495] T 1c670ba19f304e979a9405aadd4c0aca P 374d1f9d096c4e89b211c84c4228b9ae reported cstate change: term changed from 0 to 1, leader changed from <none> to 374d1f9d096c4e89b211c84c4228b9ae (127.1.192.194). New cstate: current_term: 1 leader_uuid: "374d1f9d096c4e89b211c84c4228b9ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "350e9b1bbad84279bf35b4e4db434b6b" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 40369 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "374d1f9d096c4e89b211c84c4228b9ae" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 40381 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "905d1a29a3c4431499499b3df47ecaf4" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 42657 } health_report { overall_health: UNKNOWN } } }
Aug 09 02:02:30 dist-test-slave-ss4n krb5kdc[2943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240948, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.254@KRBTEST.COM for kudu/127.1.192.254@KRBTEST.COM
I20260809 02:02:30.144883 3574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:30.135531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:39235 (local address 127.0.0.1:43810)
0809 02:02:30.135856 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:30.135870 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 02:02:30.135985 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:02:30.136250 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:02:30.136252 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:02:30.136691 (+ 439us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:02:30.136792 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:30.136799 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:30.137654 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:30.137657 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:30.138268 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:30.138276 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:30.138764 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:30.138766 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:30.138834 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:30.139485 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:02:30.139498 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:02:30.141152 (+ 1654us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:02:30.143202 (+ 2050us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:30.143215 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:30.143222 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:30.143461 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:30.143755 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:30.143757 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:30.143759 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:30.143843 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:30.144227 (+ 384us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:02:30.144236 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:02:30.144372 (+ 136us) client_negotiation.cc:769] Sending connection context
0809 02:02:30.144500 (+ 128us) client_negotiation.cc:240] Negotiation successful
0809 02:02:30.144672 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":119,"threads_started":1}
I20260809 02:02:30.144883 3575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:30.135542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43810 (local address 127.1.192.254:39235)
0809 02:02:30.135866 (+ 324us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:30.135878 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:30.135946 (+ 68us) server_negotiation.cc:407] Connection header received
0809 02:02:30.136075 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:30.136077 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:30.136124 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:30.136211 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:30.136908 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:30.137527 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:30.138384 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:30.138655 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:30.138741 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:30.141307 (+ 2566us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:02:30.141310 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:02:30.141312 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:02:30.141331 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:02:30.143085 (+ 1754us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:30.143562 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:30.143568 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:30.143570 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:30.143656 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:30.143931 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:30.143936 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:30.143937 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:30.144120 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:02:30.144227 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:30.144558 (+ 331us) server_negotiation.cc:299] Negotiation successful
0809 02:02:30.144730 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":68,"server-negotiator.queue_time_us":247,"thread_start_us":130,"threads_started":1}
I20260809 02:02:30.149838 2992 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.1.192.254@KRBTEST.COM'} at 127.0.0.1:43810:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 02:02:30.150661 2992 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:02:30.156075 3440 tablet_service.cc:1467] Processing CreateTablet for tablet 77e4b080e366423da2846fd58ecbedc6 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a1a74777b9af496199174fbe1030f3f6]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 02:02:30.156659 3440 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77e4b080e366423da2846fd58ecbedc6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:02:30.156136 3291 tablet_service.cc:1467] Processing CreateTablet for tablet 77e4b080e366423da2846fd58ecbedc6 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a1a74777b9af496199174fbe1030f3f6]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 02:02:30.158957 3291 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77e4b080e366423da2846fd58ecbedc6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:02:30.156059 3136 tablet_service.cc:1467] Processing CreateTablet for tablet 77e4b080e366423da2846fd58ecbedc6 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a1a74777b9af496199174fbe1030f3f6]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 02:02:30.159505 3136 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77e4b080e366423da2846fd58ecbedc6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:02:30.159884 3536 tablet_bootstrap.cc:492] T 77e4b080e366423da2846fd58ecbedc6 P 905d1a29a3c4431499499b3df47ecaf4: Bootstrap starting.
I20260809 02:02:30.160867 3536 tablet_bootstrap.cc:654] T 77e4b080e366423da2846fd58ecbedc6 P 905d1a29a3c4431499499b3df47ecaf4: Neither blocks nor log segments found. Creating new log.
I20260809 02:02:30.162937 3535 tablet_bootstrap.cc:492] T 77e4b080e366423da2846fd58ecbedc6 P 350e9b1bbad84279bf35b4e4db434b6b: Bootstrap starting.
I20260809 02:02:30.163798 3535 tablet_bootstrap.cc:654] T 77e4b080e366423da2846fd58ecbedc6 P 350e9b1bbad84279bf35b4e4db434b6b: Neither blocks nor log segments found. Creating new log.
I20260809 02:02:30.164806 3536 tablet_bootstrap.cc:492] T 77e4b080e366423da2846fd58ecbedc6 P 905d1a29a3c4431499499b3df47ecaf4: No bootstrap required, opened a new log
I20260809 02:02:30.164932 3536 ts_tablet_manager.cc:1397] T 77e4b080e366423da2846fd58ecbedc6 P 905d1a29a3c4431499499b3df47ecaf4: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260809 02:02:30.165223 3535 tablet_bootstrap.cc:492] T 77e4b080e366423da2846fd58ecbedc6 P 350e9b1bbad84279bf35b4e4db434b6b: No bootstrap required, opened a new log
I20260809 02:02:30.165341 3535 ts_tablet_manager.cc:1397] T 77e4b080e366423da2846fd58ecbedc6 P 350e9b1bbad84279bf35b4e4db434b6b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 02:02:30.165362 3536 raft_consensus.cc:348] T 77e4b080e366423da2846fd58ecbedc6 P 905d1a29a3c4431499499b3df47ecaf4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "350e9b1bbad84279bf35b4e4db434b6b" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 40369 } } peers { permanent_uuid: "374d1f9d096c4e89b211c84c4228b9ae" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 40381 } } peers { permanent_uuid: "905d1a29a3c4431499499b3df47ecaf4" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 42657 } }
I20260809 02:02:30.165534 3536 raft_consensus.cc:374] T 77e4b080e366423da2846fd58ecbedc6 P 905d1a29a3c4431499499b3df47ecaf4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:02:30.165580 3536 raft_consensus.cc:729] T 77e4b080e366423da2846fd58ecbedc6 P 905d1a29a3c4431499499b3df47ecaf4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 905d1a29a3c4431499499b3df47ecaf4, State: Initialized, Role: FOLLOWER
I20260809 02:02:30.165726 3536 consensus_queue.cc:260] T 77e4b080e366423da2846fd58ecbedc6 P 905d1a29a3c4431499499b3df47ecaf4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "350e9b1bbad84279bf35b4e4db434b6b" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 40369 } } peers { permanent_uuid: "374d1f9d096c4e89b211c84c4228b9ae" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 40381 } } peers { permanent_uuid: "905d1a29a3c4431499499b3df47ecaf4" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 42657 } }
I20260809 02:02:30.165934 3535 raft_consensus.cc:348] T 77e4b080e366423da2846fd58ecbedc6 P 350e9b1bbad84279bf35b4e4db434b6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "350e9b1bbad84279bf35b4e4db434b6b" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 40369 } } peers { permanent_uuid: "374d1f9d096c4e89b211c84c4228b9ae" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 40381 } } peers { permanent_uuid: "905d1a29a3c4431499499b3df47ecaf4" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 42657 } }
I20260809 02:02:30.166077 3535 raft_consensus.cc:374] T 77e4b080e366423da2846fd58ecbedc6 P 350e9b1bbad84279bf35b4e4db434b6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:02:30.166059 3554 tablet_replica.cc:439] T 77e4b080e366423da2846fd58ecbedc6 P 905d1a29a3c4431499499b3df47ecaf4: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "350e9b1bbad84279bf35b4e4db434b6b" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 40369 } } peers { permanent_uuid: "374d1f9d096c4e89b211c84c4228b9ae" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 40381 } } peers { permanent_uuid: "905d1a29a3c4431499499b3df47ecaf4" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 42657 } } }
I20260809 02:02:30.166121 3535 raft_consensus.cc:729] T 77e4b080e366423da2846fd58ecbedc6 P 350e9b1bbad84279bf35b4e4db434b6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 350e9b1bbad84279bf35b4e4db434b6b, State: Initialized, Role: FOLLOWER
I20260809 02:02:30.166196 3554 tablet_replica.cc:442] T 77e4b080e366423da2846fd58ecbedc6 P 905d1a29a3c4431499499b3df47ecaf4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 02:02:30.166311 3535 consensus_queue.cc:260] T 77e4b080e366423da2846fd58ecbedc6 P 350e9b1bbad84279bf35b4e4db434b6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "350e9b1bbad84279bf35b4e4db434b6b" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 40369 } } peers { permanent_uuid: "374d1f9d096c4e89b211c84c4228b9ae" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 40381 } } peers { permanent_uuid: "905d1a29a3c4431499499b3df47ecaf4" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 42657 } }
I20260809 02:02:30.166605 3544 tablet_replica.cc:439] T 77e4b080e366423da2846fd58ecbedc6 P 350e9b1bbad84279bf35b4e4db434b6b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "350e9b1bbad84279bf35b4e4db434b6b" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 40369 } } peers { permanent_uuid: "374d1f9d096c4e89b211c84c4228b9ae" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 40381 } } peers { permanent_uuid: "905d1a29a3c4431499499b3df47ecaf4" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 42657 } } }
I20260809 02:02:30.166693 3535 ts_tablet_manager.cc:1428] T 77e4b080e366423da2846fd58ecbedc6 P 350e9b1bbad84279bf35b4e4db434b6b: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260809 02:02:30.166777 3544 tablet_replica.cc:442] T 77e4b080e366423da2846fd58ecbedc6 P 350e9b1bbad84279bf35b4e4db434b6b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 02:02:30.167181 3536 ts_tablet_manager.cc:1428] T 77e4b080e366423da2846fd58ecbedc6 P 905d1a29a3c4431499499b3df47ecaf4: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260809 02:02:30.169297 3538 tablet_bootstrap.cc:492] T 77e4b080e366423da2846fd58ecbedc6 P 374d1f9d096c4e89b211c84c4228b9ae: Bootstrap starting.
I20260809 02:02:30.170521 3538 tablet_bootstrap.cc:654] T 77e4b080e366423da2846fd58ecbedc6 P 374d1f9d096c4e89b211c84c4228b9ae: Neither blocks nor log segments found. Creating new log.
I20260809 02:02:30.171792 3538 tablet_bootstrap.cc:492] T 77e4b080e366423da2846fd58ecbedc6 P 374d1f9d096c4e89b211c84c4228b9ae: No bootstrap required, opened a new log
I20260809 02:02:30.171881 3538 ts_tablet_manager.cc:1397] T 77e4b080e366423da2846fd58ecbedc6 P 374d1f9d096c4e89b211c84c4228b9ae: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 02:02:30.172307 3538 raft_consensus.cc:348] T 77e4b080e366423da2846fd58ecbedc6 P 374d1f9d096c4e89b211c84c4228b9ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "350e9b1bbad84279bf35b4e4db434b6b" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 40369 } } peers { permanent_uuid: "374d1f9d096c4e89b211c84c4228b9ae" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 40381 } } peers { permanent_uuid: "905d1a29a3c4431499499b3df47ecaf4" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 42657 } }
I20260809 02:02:30.172418 3538 raft_consensus.cc:374] T 77e4b080e366423da2846fd58ecbedc6 P 374d1f9d096c4e89b211c84c4228b9ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:02:30.172451 3538 raft_consensus.cc:729] T 77e4b080e366423da2846fd58ecbedc6 P 374d1f9d096c4e89b211c84c4228b9ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 374d1f9d096c4e89b211c84c4228b9ae, State: Initialized, Role: FOLLOWER
I20260809 02:02:30.172567 3538 consensus_queue.cc:260] T 77e4b080e366423da2846fd58ecbedc6 P 374d1f9d096c4e89b211c84c4228b9ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "350e9b1bbad84279bf35b4e4db434b6b" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 40369 } } peers { permanent_uuid: "374d1f9d096c4e89b211c84c4228b9ae" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 40381 } } peers { permanent_uuid: "905d1a29a3c4431499499b3df47ecaf4" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 42657 } }
I20260809 02:02:30.172847 3542 tablet_replica.cc:439] T 77e4b080e366423da2846fd58ecbedc6 P 374d1f9d096c4e89b211c84c4228b9ae: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "350e9b1bbad84279bf35b4e4db434b6b" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 40369 } } peers { permanent_uuid: "374d1f9d096c4e89b211c84c4228b9ae" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 40381 } } peers { permanent_uuid: "905d1a29a3c4431499499b3df47ecaf4" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 42657 } } }
I20260809 02:02:30.172911 3538 ts_tablet_manager.cc:1428] T 77e4b080e366423da2846fd58ecbedc6 P 374d1f9d096c4e89b211c84c4228b9ae: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260809 02:02:30.172995 3542 tablet_replica.cc:442] T 77e4b080e366423da2846fd58ecbedc6 P 374d1f9d096c4e89b211c84c4228b9ae: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 02:02:30.248246 3542 raft_consensus.cc:482] T 77e4b080e366423da2846fd58ecbedc6 P 374d1f9d096c4e89b211c84c4228b9ae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:02:30.248420 3542 raft_consensus.cc:504] T 77e4b080e366423da2846fd58ecbedc6 P 374d1f9d096c4e89b211c84c4228b9ae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "350e9b1bbad84279bf35b4e4db434b6b" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 40369 } } peers { permanent_uuid: "374d1f9d096c4e89b211c84c4228b9ae" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 40381 } } peers { permanent_uuid: "905d1a29a3c4431499499b3df47ecaf4" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 42657 } }
I20260809 02:02:30.248911 3542 leader_election.cc:288] T 77e4b080e366423da2846fd58ecbedc6 P 374d1f9d096c4e89b211c84c4228b9ae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 350e9b1bbad84279bf35b4e4db434b6b (127.1.192.193:40369), 905d1a29a3c4431499499b3df47ecaf4 (127.1.192.195:42657)
I20260809 02:02:30.249441 3157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77e4b080e366423da2846fd58ecbedc6" candidate_uuid: "374d1f9d096c4e89b211c84c4228b9ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "350e9b1bbad84279bf35b4e4db434b6b" is_pre_election: true
I20260809 02:02:30.249656 3157 raft_consensus.cc:2454] T 77e4b080e366423da2846fd58ecbedc6 P 350e9b1bbad84279bf35b4e4db434b6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 374d1f9d096c4e89b211c84c4228b9ae in term 0.
I20260809 02:02:30.249666 3460 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77e4b080e366423da2846fd58ecbedc6" candidate_uuid: "374d1f9d096c4e89b211c84c4228b9ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "905d1a29a3c4431499499b3df47ecaf4" is_pre_election: true
I20260809 02:02:30.249800 3460 raft_consensus.cc:2454] T 77e4b080e366423da2846fd58ecbedc6 P 905d1a29a3c4431499499b3df47ecaf4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 374d1f9d096c4e89b211c84c4228b9ae in term 0.
I20260809 02:02:30.250032 3240 leader_election.cc:302] T 77e4b080e366423da2846fd58ecbedc6 P 374d1f9d096c4e89b211c84c4228b9ae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 350e9b1bbad84279bf35b4e4db434b6b, 374d1f9d096c4e89b211c84c4228b9ae; no voters:
I20260809 02:02:30.250254 3542 raft_consensus.cc:2781] T 77e4b080e366423da2846fd58ecbedc6 P 374d1f9d096c4e89b211c84c4228b9ae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:02:30.250335 3542 raft_consensus.cc:482] T 77e4b080e366423da2846fd58ecbedc6 P 374d1f9d096c4e89b211c84c4228b9ae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:02:30.250391 3542 raft_consensus.cc:3037] T 77e4b080e366423da2846fd58ecbedc6 P 374d1f9d096c4e89b211c84c4228b9ae [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:02:30.251384 3542 raft_consensus.cc:504] T 77e4b080e366423da2846fd58ecbedc6 P 374d1f9d096c4e89b211c84c4228b9ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "350e9b1bbad84279bf35b4e4db434b6b" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 40369 } } peers { permanent_uuid: "374d1f9d096c4e89b211c84c4228b9ae" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 40381 } } peers { permanent_uuid: "905d1a29a3c4431499499b3df47ecaf4" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 42657 } }
I20260809 02:02:30.251730 3542 leader_election.cc:288] T 77e4b080e366423da2846fd58ecbedc6 P 374d1f9d096c4e89b211c84c4228b9ae [CANDIDATE]: Term 1 election: Requested vote from peers 350e9b1bbad84279bf35b4e4db434b6b (127.1.192.193:40369), 905d1a29a3c4431499499b3df47ecaf4 (127.1.192.195:42657)
I20260809 02:02:30.252113 3157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77e4b080e366423da2846fd58ecbedc6" candidate_uuid: "374d1f9d096c4e89b211c84c4228b9ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "350e9b1bbad84279bf35b4e4db434b6b"
I20260809 02:02:30.252121 3460 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77e4b080e366423da2846fd58ecbedc6" candidate_uuid: "374d1f9d096c4e89b211c84c4228b9ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "905d1a29a3c4431499499b3df47ecaf4"
I20260809 02:02:30.252243 3157 raft_consensus.cc:3037] T 77e4b080e366423da2846fd58ecbedc6 P 350e9b1bbad84279bf35b4e4db434b6b [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:02:30.252243 3460 raft_consensus.cc:3037] T 77e4b080e366423da2846fd58ecbedc6 P 905d1a29a3c4431499499b3df47ecaf4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:02:30.253394 3460 raft_consensus.cc:2454] T 77e4b080e366423da2846fd58ecbedc6 P 905d1a29a3c4431499499b3df47ecaf4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 374d1f9d096c4e89b211c84c4228b9ae in term 1.
I20260809 02:02:30.253394 3157 raft_consensus.cc:2454] T 77e4b080e366423da2846fd58ecbedc6 P 350e9b1bbad84279bf35b4e4db434b6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 374d1f9d096c4e89b211c84c4228b9ae in term 1.
I20260809 02:02:30.253755 3240 leader_election.cc:302] T 77e4b080e366423da2846fd58ecbedc6 P 374d1f9d096c4e89b211c84c4228b9ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 350e9b1bbad84279bf35b4e4db434b6b, 374d1f9d096c4e89b211c84c4228b9ae; no voters:
I20260809 02:02:30.253940 3542 raft_consensus.cc:2781] T 77e4b080e366423da2846fd58ecbedc6 P 374d1f9d096c4e89b211c84c4228b9ae [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:02:30.254038 3542 raft_consensus.cc:686] T 77e4b080e366423da2846fd58ecbedc6 P 374d1f9d096c4e89b211c84c4228b9ae [term 1 LEADER]: Becoming Leader. State: Replica: 374d1f9d096c4e89b211c84c4228b9ae, State: Running, Role: LEADER
I20260809 02:02:30.254175 3542 consensus_queue.cc:237] T 77e4b080e366423da2846fd58ecbedc6 P 374d1f9d096c4e89b211c84c4228b9ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "350e9b1bbad84279bf35b4e4db434b6b" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 40369 } } peers { permanent_uuid: "374d1f9d096c4e89b211c84c4228b9ae" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 40381 } } peers { permanent_uuid: "905d1a29a3c4431499499b3df47ecaf4" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 42657 } }
I20260809 02:02:30.254863 3567 tablet_replica.cc:439] T 77e4b080e366423da2846fd58ecbedc6 P 374d1f9d096c4e89b211c84c4228b9ae: TxnStatusTablet state changed. Reason: New leader 374d1f9d096c4e89b211c84c4228b9ae. Latest consensus state: current_term: 1 leader_uuid: "374d1f9d096c4e89b211c84c4228b9ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "350e9b1bbad84279bf35b4e4db434b6b" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 40369 } } peers { permanent_uuid: "374d1f9d096c4e89b211c84c4228b9ae" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 40381 } } peers { permanent_uuid: "905d1a29a3c4431499499b3df47ecaf4" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 42657 } } }
I20260809 02:02:30.254977 3567 tablet_replica.cc:442] T 77e4b080e366423da2846fd58ecbedc6 P 374d1f9d096c4e89b211c84c4228b9ae: This TxnStatusTablet replica's current role is: LEADER
I20260809 02:02:30.255364 3580 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 02:02:30.255578 3580 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 02:02:30.256100 2987 catalog_manager.cc:5495] T 77e4b080e366423da2846fd58ecbedc6 P 374d1f9d096c4e89b211c84c4228b9ae reported cstate change: term changed from 0 to 1, leader changed from <none> to 374d1f9d096c4e89b211c84c4228b9ae (127.1.192.194). New cstate: current_term: 1 leader_uuid: "374d1f9d096c4e89b211c84c4228b9ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "350e9b1bbad84279bf35b4e4db434b6b" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 40369 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "374d1f9d096c4e89b211c84c4228b9ae" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 40381 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "905d1a29a3c4431499499b3df47ecaf4" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 42657 } health_report { overall_health: UNKNOWN } } }
I20260809 02:02:30.288692 3574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:30.284562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.194:40381 (local address 127.0.0.1:45032)
0809 02:02:30.284724 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:30.284742 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 02:02:30.284860 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:02:30.285318 (+ 458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:02:30.285321 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:02:30.285329 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 02:02:30.285446 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:30.285451 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:30.286249 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:30.286253 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:30.287021 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:30.287030 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:30.287536 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:30.287540 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:30.287630 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:30.287674 (+ 44us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 02:02:30.288380 (+ 706us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 02:02:30.288396 (+ 16us) client_negotiation.cc:769] Sending connection context
0809 02:02:30.288463 (+ 67us) client_negotiation.cc:240] Negotiation successful
0809 02:02:30.288546 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260809 02:02:30.288710 3584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:30.284706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45032 (local address 127.1.192.194:40381)
0809 02:02:30.285044 (+ 338us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:30.285051 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:30.285063 (+ 12us) server_negotiation.cc:407] Connection header received
0809 02:02:30.285131 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:30.285133 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:30.285184 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:30.285304 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:02:30.285604 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:30.286135 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:30.287148 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:30.287398 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:30.287512 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:30.287847 (+ 335us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:02:30.288269 (+ 422us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:02:30.288395 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:30.288474 (+ 79us) server_negotiation.cc:299] Negotiation successful
0809 02:02:30.288527 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":256,"thread_start_us":108,"threads_started":1}
I20260809 02:02:30.291859 3157 raft_consensus.cc:1261] T 77e4b080e366423da2846fd58ecbedc6 P 350e9b1bbad84279bf35b4e4db434b6b [term 1 FOLLOWER]: Refusing update from remote peer 374d1f9d096c4e89b211c84c4228b9ae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:02:30.292407 3542 consensus_queue.cc:1035] T 77e4b080e366423da2846fd58ecbedc6 P 374d1f9d096c4e89b211c84c4228b9ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "350e9b1bbad84279bf35b4e4db434b6b" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 40369 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:02:30.292484 3460 raft_consensus.cc:1261] T 77e4b080e366423da2846fd58ecbedc6 P 905d1a29a3c4431499499b3df47ecaf4 [term 1 FOLLOWER]: Refusing update from remote peer 374d1f9d096c4e89b211c84c4228b9ae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:02:30.293118 3542 consensus_queue.cc:1035] T 77e4b080e366423da2846fd58ecbedc6 P 374d1f9d096c4e89b211c84c4228b9ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "905d1a29a3c4431499499b3df47ecaf4" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 42657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:02:30.294026 3544 tablet_replica.cc:439] T 77e4b080e366423da2846fd58ecbedc6 P 350e9b1bbad84279bf35b4e4db434b6b: TxnStatusTablet state changed. Reason: New leader 374d1f9d096c4e89b211c84c4228b9ae. Latest consensus state: current_term: 1 leader_uuid: "374d1f9d096c4e89b211c84c4228b9ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "350e9b1bbad84279bf35b4e4db434b6b" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 40369 } } peers { permanent_uuid: "374d1f9d096c4e89b211c84c4228b9ae" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 40381 } } peers { permanent_uuid: "905d1a29a3c4431499499b3df47ecaf4" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 42657 } } }
I20260809 02:02:30.294155 3544 tablet_replica.cc:442] T 77e4b080e366423da2846fd58ecbedc6 P 350e9b1bbad84279bf35b4e4db434b6b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 02:02:30.296837 3554 tablet_replica.cc:439] T 77e4b080e366423da2846fd58ecbedc6 P 905d1a29a3c4431499499b3df47ecaf4: TxnStatusTablet state changed. Reason: New leader 374d1f9d096c4e89b211c84c4228b9ae. Latest consensus state: current_term: 1 leader_uuid: "374d1f9d096c4e89b211c84c4228b9ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "350e9b1bbad84279bf35b4e4db434b6b" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 40369 } } peers { permanent_uuid: "374d1f9d096c4e89b211c84c4228b9ae" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 40381 } } peers { permanent_uuid: "905d1a29a3c4431499499b3df47ecaf4" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 42657 } } }
I20260809 02:02:30.296967 3554 tablet_replica.cc:442] T 77e4b080e366423da2846fd58ecbedc6 P 905d1a29a3c4431499499b3df47ecaf4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 02:02:30.298091 3567 tablet_replica.cc:439] T 77e4b080e366423da2846fd58ecbedc6 P 374d1f9d096c4e89b211c84c4228b9ae: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "374d1f9d096c4e89b211c84c4228b9ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "350e9b1bbad84279bf35b4e4db434b6b" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 40369 } } peers { permanent_uuid: "374d1f9d096c4e89b211c84c4228b9ae" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 40381 } } peers { permanent_uuid: "905d1a29a3c4431499499b3df47ecaf4" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 42657 } } }
I20260809 02:02:30.298225 3567 tablet_replica.cc:442] T 77e4b080e366423da2846fd58ecbedc6 P 374d1f9d096c4e89b211c84c4228b9ae: This TxnStatusTablet replica's current role is: LEADER
I20260809 02:02:30.299069 3544 tablet_replica.cc:439] T 77e4b080e366423da2846fd58ecbedc6 P 350e9b1bbad84279bf35b4e4db434b6b: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "374d1f9d096c4e89b211c84c4228b9ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "350e9b1bbad84279bf35b4e4db434b6b" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 40369 } } peers { permanent_uuid: "374d1f9d096c4e89b211c84c4228b9ae" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 40381 } } peers { permanent_uuid: "905d1a29a3c4431499499b3df47ecaf4" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 42657 } } }
I20260809 02:02:30.299191 3544 tablet_replica.cc:442] T 77e4b080e366423da2846fd58ecbedc6 P 350e9b1bbad84279bf35b4e4db434b6b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 02:02:30.301355 3567 tablet_replica.cc:439] T 77e4b080e366423da2846fd58ecbedc6 P 374d1f9d096c4e89b211c84c4228b9ae: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "374d1f9d096c4e89b211c84c4228b9ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "350e9b1bbad84279bf35b4e4db434b6b" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 40369 } } peers { permanent_uuid: "374d1f9d096c4e89b211c84c4228b9ae" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 40381 } } peers { permanent_uuid: "905d1a29a3c4431499499b3df47ecaf4" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 42657 } } }
I20260809 02:02:30.301570 3567 tablet_replica.cc:442] T 77e4b080e366423da2846fd58ecbedc6 P 374d1f9d096c4e89b211c84c4228b9ae: This TxnStatusTablet replica's current role is: LEADER
I20260809 02:02:30.303176 3554 tablet_replica.cc:439] T 77e4b080e366423da2846fd58ecbedc6 P 905d1a29a3c4431499499b3df47ecaf4: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "374d1f9d096c4e89b211c84c4228b9ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "350e9b1bbad84279bf35b4e4db434b6b" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 40369 } } peers { permanent_uuid: "374d1f9d096c4e89b211c84c4228b9ae" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 40381 } } peers { permanent_uuid: "905d1a29a3c4431499499b3df47ecaf4" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 42657 } } }
I20260809 02:02:30.303298 3554 tablet_replica.cc:442] T 77e4b080e366423da2846fd58ecbedc6 P 905d1a29a3c4431499499b3df47ecaf4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 02:02:30.305600 3588 mvcc.cc:205] Tried to move back new op lower bound from 7316442932390252544 to 7316442932242530304. Current Snapshot: MvccSnapshot[applied={T|T < 7316442932390252544}]
I20260809 02:02:30.307950 3584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:30.304197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45048 (local address 127.1.192.194:40381)
0809 02:02:30.304441 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:30.304448 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:30.304477 (+ 29us) server_negotiation.cc:407] Connection header received
0809 02:02:30.304625 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:30.304629 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:30.304687 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:30.304780 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:02:30.305029 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:30.305649 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:30.306761 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:30.307124 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:30.307231 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:30.307455 (+ 224us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:02:30.307536 (+ 81us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:02:30.307624 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:30.307743 (+ 119us) server_negotiation.cc:299] Negotiation successful
0809 02:02:30.307795 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170}
I20260809 02:02:30.319008 3584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:30.315059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.194:43389 (local address 127.1.192.194:40381)
0809 02:02:30.315194 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:30.315199 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:30.315418 (+ 219us) server_negotiation.cc:407] Connection header received
0809 02:02:30.315601 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:30.315604 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:30.315647 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:30.315735 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:02:30.316018 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:30.316504 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:30.317785 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:30.317991 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:30.318069 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:30.318458 (+ 389us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:02:30.318573 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:02:30.318691 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:30.318811 (+ 120us) server_negotiation.cc:299] Negotiation successful
0809 02:02:30.318888 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":20,"server-negotiator.queue_time_us":56}
I20260809 02:02:30.319223 3598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:30.315006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.194:40381 (local address 127.1.192.194:43389)
0809 02:02:30.315324 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:30.315339 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 02:02:30.315447 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:02:30.315757 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:02:30.315760 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:02:30.315789 (+ 29us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 02:02:30.315914 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:30.315919 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:30.316610 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:30.316613 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:30.317660 (+ 1047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:30.317671 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:30.318108 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:30.318149 (+ 41us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:30.318258 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:30.318310 (+ 52us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 02:02:30.318715 (+ 405us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 02:02:30.318721 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 02:02:30.318788 (+ 67us) client_negotiation.cc:240] Negotiation successful
0809 02:02:30.318883 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":217,"mutex_wait_us":3,"thread_start_us":124,"threads_started":1}
I20260809 02:02:30.326449 3460 raft_consensus.cc:1261] T 1c670ba19f304e979a9405aadd4c0aca P 905d1a29a3c4431499499b3df47ecaf4 [term 1 FOLLOWER]: Refusing update from remote peer 374d1f9d096c4e89b211c84c4228b9ae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:02:30.326584 3157 raft_consensus.cc:1261] T 1c670ba19f304e979a9405aadd4c0aca P 350e9b1bbad84279bf35b4e4db434b6b [term 1 FOLLOWER]: Refusing update from remote peer 374d1f9d096c4e89b211c84c4228b9ae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:02:30.326866 3567 consensus_queue.cc:1035] T 1c670ba19f304e979a9405aadd4c0aca P 374d1f9d096c4e89b211c84c4228b9ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "905d1a29a3c4431499499b3df47ecaf4" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 42657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:02:30.327106 3542 consensus_queue.cc:1035] T 1c670ba19f304e979a9405aadd4c0aca P 374d1f9d096c4e89b211c84c4228b9ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "350e9b1bbad84279bf35b4e4db434b6b" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 40369 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:02:30.376216 2991 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43800:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:02:30.376389 2991 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43800:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:02:30.378824 2991 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 484e26ab9179442793b9906de0294663: Sending DeleteTablet for 3 replicas of tablet 1c670ba19f304e979a9405aadd4c0aca
I20260809 02:02:30.379532 3291 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c670ba19f304e979a9405aadd4c0aca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:02:30 UTC) from {username='kudu', principal='kudu/127.1.192.254@KRBTEST.COM'} at 127.0.0.1:45016
I20260809 02:02:30.379542 3136 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c670ba19f304e979a9405aadd4c0aca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:02:30 UTC) from {username='kudu', principal='kudu/127.1.192.254@KRBTEST.COM'} at 127.0.0.1:45388
I20260809 02:02:30.379959 3602 tablet_replica.cc:330] T 1c670ba19f304e979a9405aadd4c0aca P 374d1f9d096c4e89b211c84c4228b9ae: stopping tablet replica
I20260809 02:02:30.380096 3602 raft_consensus.cc:2229] T 1c670ba19f304e979a9405aadd4c0aca P 374d1f9d096c4e89b211c84c4228b9ae [term 1 LEADER]: Raft consensus shutting down.
I20260809 02:02:30.380463 3602 raft_consensus.cc:2258] T 1c670ba19f304e979a9405aadd4c0aca P 374d1f9d096c4e89b211c84c4228b9ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:02:30.381394 3602 ts_tablet_manager.cc:1905] T 1c670ba19f304e979a9405aadd4c0aca P 374d1f9d096c4e89b211c84c4228b9ae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:02:30.385030 3602 ts_tablet_manager.cc:1918] T 1c670ba19f304e979a9405aadd4c0aca P 374d1f9d096c4e89b211c84c4228b9ae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 02:02:30.385138 3602 log.cc:1196] T 1c670ba19f304e979a9405aadd4c0aca P 374d1f9d096c4e89b211c84c4228b9ae: Deleting WAL directory at /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786240941968412-1795-0/minicluster-data/ts-1/wal/wals/1c670ba19f304e979a9405aadd4c0aca
I20260809 02:02:30.385346 3440 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c670ba19f304e979a9405aadd4c0aca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:02:30 UTC) from {username='kudu', principal='kudu/127.1.192.254@KRBTEST.COM'} at 127.0.0.1:60718
I20260809 02:02:30.385432 3602 ts_tablet_manager.cc:1939] T 1c670ba19f304e979a9405aadd4c0aca P 374d1f9d096c4e89b211c84c4228b9ae: Deleting consensus metadata
I20260809 02:02:30.385661 3603 tablet_replica.cc:330] T 1c670ba19f304e979a9405aadd4c0aca P 350e9b1bbad84279bf35b4e4db434b6b: stopping tablet replica
I20260809 02:02:30.385838 3603 raft_consensus.cc:2229] T 1c670ba19f304e979a9405aadd4c0aca P 350e9b1bbad84279bf35b4e4db434b6b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:02:30.386034 3603 raft_consensus.cc:2258] T 1c670ba19f304e979a9405aadd4c0aca P 350e9b1bbad84279bf35b4e4db434b6b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:02:30.386282 2975 catalog_manager.cc:4841] TS 374d1f9d096c4e89b211c84c4228b9ae (127.1.192.194:40381): tablet 1c670ba19f304e979a9405aadd4c0aca (table test-table [id=12fa81fec6a946979a23a9dcf25054a9]) successfully deleted
I20260809 02:02:30.386955 3606 tablet_replica.cc:330] T 1c670ba19f304e979a9405aadd4c0aca P 905d1a29a3c4431499499b3df47ecaf4: stopping tablet replica
I20260809 02:02:30.387131 3606 raft_consensus.cc:2229] T 1c670ba19f304e979a9405aadd4c0aca P 905d1a29a3c4431499499b3df47ecaf4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:02:30.387197 3603 ts_tablet_manager.cc:1905] T 1c670ba19f304e979a9405aadd4c0aca P 350e9b1bbad84279bf35b4e4db434b6b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:02:30.387306 3606 raft_consensus.cc:2258] T 1c670ba19f304e979a9405aadd4c0aca P 905d1a29a3c4431499499b3df47ecaf4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:02:30.388127 3606 ts_tablet_manager.cc:1905] T 1c670ba19f304e979a9405aadd4c0aca P 905d1a29a3c4431499499b3df47ecaf4: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 09 02:02:30 dist-test-slave-ss4n krb5kdc[2943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240949, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.192.193@KRBTEST.COM
I20260809 02:02:30.390883 3603 ts_tablet_manager.cc:1918] T 1c670ba19f304e979a9405aadd4c0aca P 350e9b1bbad84279bf35b4e4db434b6b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 02:02:30.391052 3603 log.cc:1196] T 1c670ba19f304e979a9405aadd4c0aca P 350e9b1bbad84279bf35b4e4db434b6b: Deleting WAL directory at /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786240941968412-1795-0/minicluster-data/ts-0/wal/wals/1c670ba19f304e979a9405aadd4c0aca
I20260809 02:02:30.391427 3603 ts_tablet_manager.cc:1939] T 1c670ba19f304e979a9405aadd4c0aca P 350e9b1bbad84279bf35b4e4db434b6b: Deleting consensus metadata
I20260809 02:02:30.391708 3606 ts_tablet_manager.cc:1918] T 1c670ba19f304e979a9405aadd4c0aca P 905d1a29a3c4431499499b3df47ecaf4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 02:02:30.391825 3606 log.cc:1196] T 1c670ba19f304e979a9405aadd4c0aca P 905d1a29a3c4431499499b3df47ecaf4: Deleting WAL directory at /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786240941968412-1795-0/minicluster-data/ts-2/wal/wals/1c670ba19f304e979a9405aadd4c0aca
I20260809 02:02:30.392174 3606 ts_tablet_manager.cc:1939] T 1c670ba19f304e979a9405aadd4c0aca P 905d1a29a3c4431499499b3df47ecaf4: Deleting consensus metadata
I20260809 02:02:30.392407 2979 catalog_manager.cc:4841] TS 350e9b1bbad84279bf35b4e4db434b6b (127.1.192.193:40369): tablet 1c670ba19f304e979a9405aadd4c0aca (table test-table [id=12fa81fec6a946979a23a9dcf25054a9]) successfully deleted
I20260809 02:02:30.393083 2979 catalog_manager.cc:4841] TS 905d1a29a3c4431499499b3df47ecaf4 (127.1.192.195:42657): tablet 1c670ba19f304e979a9405aadd4c0aca (table test-table [id=12fa81fec6a946979a23a9dcf25054a9]) successfully deleted
I20260809 02:02:30.395848 3528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:30.381261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45394 (local address 127.1.192.193:40369)
0809 02:02:30.381392 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:30.381397 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:30.382055 (+ 658us) server_negotiation.cc:407] Connection header received
0809 02:02:30.382310 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:30.382314 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:30.382374 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:30.382433 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:30.383524 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:30.384350 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:30.385808 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:30.386160 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:30.386245 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:30.391109 (+ 4864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:02:30.391123 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:02:30.391140 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:02:30.391174 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:02:30.393644 (+ 2470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:30.394229 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:30.394235 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:30.394236 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:30.394324 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:30.394677 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:30.394680 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:30.394682 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:30.394959 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:02:30.395212 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:30.395461 (+ 249us) server_negotiation.cc:299] Negotiation successful
0809 02:02:30.395666 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
W20260809 02:02:30.396652 3202 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45394
I20260809 02:02:30.407509 3575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:30.398864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43822 (local address 127.1.192.254:39235)
0809 02:02:30.399005 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:30.399011 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:30.399099 (+ 88us) server_negotiation.cc:407] Connection header received
0809 02:02:30.399326 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:30.399329 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:30.399418 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:30.399522 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:30.400285 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:30.400878 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:30.401818 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:30.402085 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:30.402220 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:30.403746 (+ 1526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:02:30.403750 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:02:30.403753 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:02:30.403781 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:02:30.405647 (+ 1866us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:30.406104 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:30.406108 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:30.406111 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:30.406211 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:30.406513 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:30.406517 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:30.406519 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:30.406730 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:02:30.406882 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:30.407145 (+ 263us) server_negotiation.cc:299] Negotiation successful
0809 02:02:30.407314 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
W20260809 02:02:30.407951 2991 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43822
I20260809 02:02:30.409430 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 3067
I20260809 02:02:30.419850 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 3220
I20260809 02:02:30.428802 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 3374
I20260809 02:02:30.437825 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 2958
2026-08-09T02:02:30Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (3221 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786240941968412-1795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:02:30 dist-test-slave-ss4n krb5kdc[3613](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:02:30 dist-test-slave-ss4n krb5kdc[3613](info): set up 2 sockets
Aug 09 02:02:30 dist-test-slave-ss4n krb5kdc[3613](info): commencing operation
krb5kdc: starting...
W20260809 02:02:31.555892 1795 mini_kdc.cc:121] Time spent starting KDC: real 1.092s user 0.006s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786240941968412-1795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786240941968412-1795-0/krb5kdc/test-user.keytab.
Aug 09 02:02:31 dist-test-slave-ss4n krb5kdc[3613](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240951, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:02:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:02:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.192.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.192.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.192.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab.
Entry for principal kudu/127.1.192.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab.
I20260809 02:02:31.701949 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786240941968412-1795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786240941968412-1795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786240941968412-1795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.192.254:43253
--webserver_interface=127.1.192.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab
--principal=kudu/127.1.192.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:36059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.192.254:43253
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:02:31.824887 3627 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:02:31.825318 3627 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:02:31.825409 3627 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:02:31.829841 3627 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:02:31.829943 3627 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 02:02:31.829974 3627 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:02:31.830020 3627 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:02:31.830058 3627 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:02:31.830116 3627 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:02:31.836079 3627 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:36059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786240941968412-1795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786240941968412-1795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.192.254:43253
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab
--principal=kudu/127.1.192.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.192.254:43253
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.192.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3627
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786240941968412-1795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:02:31.837327 3627 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:02:31.838501 3627 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:02:31.848085 3633 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:02:31.848085 3632 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:02:31.848093 3635 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:02:31.848289 3627 server_base.cc:1034] running on GCE node
I20260809 02:02:31.849011 3627 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:02:31.850332 3627 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:02:31.851516 3627 hybrid_clock.cc:648] HybridClock initialized: now 1786240951851484 us; error 50 us; skew 500 ppm
Aug 09 02:02:31 dist-test-slave-ss4n krb5kdc[3613](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240951, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:02:31.854869 3627 init.cc:377] Logged in from keytab as kudu/127.1.192.254@KRBTEST.COM (short username kudu)
I20260809 02:02:31.855952 3627 webserver.cc:460] Webserver started at http://127.1.192.254:44539/ using document root <none> and password file <none>
I20260809 02:02:31.856567 3627 fs_manager.cc:359] Metadata directory not provided
I20260809 02:02:31.856637 3627 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:02:31.856884 3627 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:02:31.859179 3627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786240941968412-1795-0/minicluster-data/master-0/data/instance:
uuid: "5731d70307644f7ea1a3ccabcb93f4c3"
format_stamp: "Formatted at 2026-08-09 02:02:31 on dist-test-slave-ss4n"
server_key: "002143f37af46ff75233942fa8ee3871"
server_key_iv: "83f53b49ad1d1aa53a94e445ec8e1673"
server_key_version: "encryptionkey@0"
I20260809 02:02:31.859822 3627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786240941968412-1795-0/minicluster-data/master-0/wal/instance:
uuid: "5731d70307644f7ea1a3ccabcb93f4c3"
format_stamp: "Formatted at 2026-08-09 02:02:31 on dist-test-slave-ss4n"
server_key: "002143f37af46ff75233942fa8ee3871"
server_key_iv: "83f53b49ad1d1aa53a94e445ec8e1673"
server_key_version: "encryptionkey@0"
I20260809 02:02:31.865772 3627 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260809 02:02:31.869587 3642 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:31.871457 3627 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 02:02:31.871624 3627 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786240941968412-1795-0/minicluster-data/master-0/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786240941968412-1795-0/minicluster-data/master-0/wal
uuid: "5731d70307644f7ea1a3ccabcb93f4c3"
format_stamp: "Formatted at 2026-08-09 02:02:31 on dist-test-slave-ss4n"
server_key: "002143f37af46ff75233942fa8ee3871"
server_key_iv: "83f53b49ad1d1aa53a94e445ec8e1673"
server_key_version: "encryptionkey@0"
I20260809 02:02:31.871773 3627 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786240941968412-1795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786240941968412-1795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786240941968412-1795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:02:31.903295 3627 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:02:31.906888 3627 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:02:31.907135 3627 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:02:31.916963 3627 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.254:43253
I20260809 02:02:31.916977 3709 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.254:43253 every 8 connection(s)
I20260809 02:02:31.918238 3627 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
I20260809 02:02:31.920914 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 3627
I20260809 02:02:31.921037 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786240941968412-1795-0/minicluster-data/master-0/wal/instance
I20260809 02:02:31.921346 1795 external_mini_cluster.cc:1428] Setting key 2a0b69d950de45dd7819be0582c4125b
I20260809 02:02:31.922096 3710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:02:31.930073 3710 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5731d70307644f7ea1a3ccabcb93f4c3: Bootstrap starting.
Aug 09 02:02:31 dist-test-slave-ss4n krb5kdc[3613](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240951, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.192.254@KRBTEST.COM
I20260809 02:02:31.932771 3710 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5731d70307644f7ea1a3ccabcb93f4c3: Neither blocks nor log segments found. Creating new log.
I20260809 02:02:31.934405 3710 log.cc:824] T 00000000000000000000000000000000 P 5731d70307644f7ea1a3ccabcb93f4c3: Log is configured to *not* fsync() on all Append() calls
I20260809 02:02:31.937173 3710 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5731d70307644f7ea1a3ccabcb93f4c3: No bootstrap required, opened a new log
I20260809 02:02:31.938436 3713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:31.922905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36316 (local address 127.1.192.254:43253)
0809 02:02:31.923412 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:31.923436 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:31.923497 (+ 61us) server_negotiation.cc:407] Connection header received
0809 02:02:31.924434 (+ 937us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:31.924468 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:31.924874 (+ 406us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:31.925239 (+ 365us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:31.926220 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:31.927698 (+ 1478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:31.928945 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:31.929661 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:31.929831 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:31.932504 (+ 2673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:02:31.932510 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:02:31.932529 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:02:31.932567 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:02:31.934934 (+ 2367us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:31.935444 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:31.935452 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:31.935460 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:31.935617 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:31.935969 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:31.935972 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:31.935974 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:31.936519 (+ 545us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:02:31.936716 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:31.937052 (+ 336us) server_negotiation.cc:299] Negotiation successful
0809 02:02:31.937388 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":338,"thread_start_us":175,"threads_started":1}
I20260809 02:02:31.941066 3710 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5731d70307644f7ea1a3ccabcb93f4c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5731d70307644f7ea1a3ccabcb93f4c3" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 43253 } }
I20260809 02:02:31.941306 3710 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5731d70307644f7ea1a3ccabcb93f4c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:02:31.941351 3710 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5731d70307644f7ea1a3ccabcb93f4c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5731d70307644f7ea1a3ccabcb93f4c3, State: Initialized, Role: FOLLOWER
I20260809 02:02:31.941797 3710 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5731d70307644f7ea1a3ccabcb93f4c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5731d70307644f7ea1a3ccabcb93f4c3" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 43253 } }
I20260809 02:02:31.941926 3710 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5731d70307644f7ea1a3ccabcb93f4c3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:02:31.941974 3710 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5731d70307644f7ea1a3ccabcb93f4c3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:02:31.942047 3710 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5731d70307644f7ea1a3ccabcb93f4c3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:02:31.943025 3710 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5731d70307644f7ea1a3ccabcb93f4c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5731d70307644f7ea1a3ccabcb93f4c3" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 43253 } }
I20260809 02:02:31.943336 3710 leader_election.cc:302] T 00000000000000000000000000000000 P 5731d70307644f7ea1a3ccabcb93f4c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5731d70307644f7ea1a3ccabcb93f4c3; no voters:
I20260809 02:02:31.943681 3710 leader_election.cc:288] T 00000000000000000000000000000000 P 5731d70307644f7ea1a3ccabcb93f4c3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:02:31.943800 3715 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5731d70307644f7ea1a3ccabcb93f4c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:02:31.944073 3715 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5731d70307644f7ea1a3ccabcb93f4c3 [term 1 LEADER]: Becoming Leader. State: Replica: 5731d70307644f7ea1a3ccabcb93f4c3, State: Running, Role: LEADER
I20260809 02:02:31.944382 3715 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5731d70307644f7ea1a3ccabcb93f4c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5731d70307644f7ea1a3ccabcb93f4c3" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 43253 } }
I20260809 02:02:31.944993 3710 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5731d70307644f7ea1a3ccabcb93f4c3 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:02:31.946560 3718 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5731d70307644f7ea1a3ccabcb93f4c3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5731d70307644f7ea1a3ccabcb93f4c3. Latest consensus state: current_term: 1 leader_uuid: "5731d70307644f7ea1a3ccabcb93f4c3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5731d70307644f7ea1a3ccabcb93f4c3" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 43253 } } }
I20260809 02:02:31.946720 3718 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5731d70307644f7ea1a3ccabcb93f4c3 [sys.catalog]: This master's current role is: LEADER
I20260809 02:02:31.947024 3716 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5731d70307644f7ea1a3ccabcb93f4c3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5731d70307644f7ea1a3ccabcb93f4c3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5731d70307644f7ea1a3ccabcb93f4c3" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 43253 } } }
I20260809 02:02:31.947147 3716 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5731d70307644f7ea1a3ccabcb93f4c3 [sys.catalog]: This master's current role is: LEADER
W20260809 02:02:31.950500 3732 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5731d70307644f7ea1a3ccabcb93f4c3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 02:02:31.950639 3732 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 02:02:31.950742 3728 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:02:31.951787 3728 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:02:31.957813 3728 catalog_manager.cc:1347] Generated new cluster ID: 935a58d7287147d09c928570ab114cde
I20260809 02:02:31.957914 3728 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:02:31.969630 3728 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:02:31.970687 3728 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:02:31.977344 3728 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5731d70307644f7ea1a3ccabcb93f4c3: Generated new TSK 0
I20260809 02:02:31.978258 3728 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.192.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.192.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.192.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.193.keytab.
Entry for principal kudu/127.1.192.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.193.keytab.
I20260809 02:02:32.018057 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786240941968412-1795-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786240941968412-1795-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786240941968412-1795-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786240941968412-1795-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.192.193:0
--local_ip_for_outbound_sockets=127.1.192.193
--webserver_interface=127.1.192.193
--webserver_port=0
--tserver_master_addrs=127.1.192.254:43253
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.193.keytab
--principal=kudu/127.1.192.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:36059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:02:32.136101 3738 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:02:32.136420 3738 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:02:32.136472 3738 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:02:32.140666 3738 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:02:32.140754 3738 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:02:32.140797 3738 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 02:02:32.140812 3738 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 02:02:32.140899 3738 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.192.193
I20260809 02:02:32.146971 3738 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:36059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786240941968412-1795-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786240941968412-1795-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.193.keytab
--principal=kudu/127.1.192.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.192.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786240941968412-1795-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.192.193
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.192.254:43253
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3738
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.192.193
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786240941968412-1795-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:02:32.148162 3738 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:02:32.149248 3738 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:02:32.159469 3745 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:02:32.159576 3747 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:02:32.159519 3743 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:02:32.160018 3738 server_base.cc:1034] running on GCE node
I20260809 02:02:32.160424 3738 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:02:32.161065 3738 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:02:32.162284 3738 hybrid_clock.cc:648] HybridClock initialized: now 1786240952162259 us; error 49 us; skew 500 ppm
Aug 09 02:02:32 dist-test-slave-ss4n krb5kdc[3613](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240952, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:02:32.166108 3738 init.cc:377] Logged in from keytab as kudu/127.1.192.193@KRBTEST.COM (short username kudu)
I20260809 02:02:32.167065 3738 webserver.cc:460] Webserver started at http://127.1.192.193:41867/ using document root <none> and password file <none>
I20260809 02:02:32.167670 3738 fs_manager.cc:359] Metadata directory not provided
I20260809 02:02:32.167732 3738 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:02:32.167955 3738 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:02:32.170173 3738 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786240941968412-1795-0/minicluster-data/ts-0/data/instance:
uuid: "3a1dc6be7d1640ba97604a9c62f597e9"
format_stamp: "Formatted at 2026-08-09 02:02:32 on dist-test-slave-ss4n"
server_key: "103d8c5a40b35db262a63b04ce92232c"
server_key_iv: "c83b5fcbc1403e5ca1a49288e6e66db4"
server_key_version: "encryptionkey@0"
I20260809 02:02:32.170758 3738 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786240941968412-1795-0/minicluster-data/ts-0/wal/instance:
uuid: "3a1dc6be7d1640ba97604a9c62f597e9"
format_stamp: "Formatted at 2026-08-09 02:02:32 on dist-test-slave-ss4n"
server_key: "103d8c5a40b35db262a63b04ce92232c"
server_key_iv: "c83b5fcbc1403e5ca1a49288e6e66db4"
server_key_version: "encryptionkey@0"
I20260809 02:02:32.174636 3738 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 02:02:32.177255 3755 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:32.178722 3738 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 02:02:32.178861 3738 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786240941968412-1795-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786240941968412-1795-0/minicluster-data/ts-0/wal
uuid: "3a1dc6be7d1640ba97604a9c62f597e9"
format_stamp: "Formatted at 2026-08-09 02:02:32 on dist-test-slave-ss4n"
server_key: "103d8c5a40b35db262a63b04ce92232c"
server_key_iv: "c83b5fcbc1403e5ca1a49288e6e66db4"
server_key_version: "encryptionkey@0"
I20260809 02:02:32.178987 3738 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786240941968412-1795-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786240941968412-1795-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786240941968412-1795-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:02:32.195806 3738 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:02:32.199079 3738 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:02:32.199249 3738 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:02:32.200867 3738 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:02:32.200953 3738 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:32.201031 3738 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:02:32.201074 3738 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 02:02:32 dist-test-slave-ss4n krb5kdc[3613](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240952, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.193@KRBTEST.COM for kudu/127.1.192.254@KRBTEST.COM
I20260809 02:02:32.212512 3738 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.193:34563
I20260809 02:02:32.212643 3879 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.193:34563 every 8 connection(s)
I20260809 02:02:32.213788 3738 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786240941968412-1795-0/minicluster-data/ts-0/data/info.pb
I20260809 02:02:32.216365 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 3738
I20260809 02:02:32.216500 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786240941968412-1795-0/minicluster-data/ts-0/wal/instance
I20260809 02:02:32.216681 3713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:32.201857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.193:52355 (local address 127.1.192.254:43253)
0809 02:02:32.202049 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:32.202054 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:32.203136 (+ 1082us) server_negotiation.cc:407] Connection header received
0809 02:02:32.204189 (+ 1053us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:32.204193 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:32.204256 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:32.204315 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:32.206304 (+ 1989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:32.206903 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:32.208106 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:32.208487 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:32.208623 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:32.212236 (+ 3613us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:02:32.212241 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:02:32.212244 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:02:32.212284 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:02:32.214188 (+ 1904us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:32.214822 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:32.214826 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:32.214828 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:32.214938 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:32.215332 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:32.215334 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:32.215335 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:32.215528 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:02:32.215662 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:32.216243 (+ 581us) server_negotiation.cc:299] Negotiation successful
0809 02:02:32.216405 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 02:02:32.216847 1795 external_mini_cluster.cc:1428] Setting key 3a17a6706a997798488c112ee4b80906
I20260809 02:02:32.217536 3766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:32.202358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:43253 (local address 127.1.192.193:52355)
0809 02:02:32.202981 (+ 623us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:32.203027 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 02:02:32.203898 (+ 871us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:02:32.204818 (+ 920us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:02:32.204828 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:02:32.205324 (+ 496us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:02:32.206136 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:32.206159 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:32.207045 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:32.207048 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:32.207957 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:32.207967 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:32.208684 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:32.208688 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:32.208875 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:32.209828 (+ 953us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:02:32.209850 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:02:32.211965 (+ 2115us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:02:32.214334 (+ 2369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:32.214341 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:32.214357 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:32.214707 (+ 350us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:32.215053 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:32.215056 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:32.215057 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:32.215243 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:32.215696 (+ 453us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:02:32.215702 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:02:32.215971 (+ 269us) client_negotiation.cc:769] Sending connection context
0809 02:02:32.216141 (+ 170us) client_negotiation.cc:240] Negotiation successful
0809 02:02:32.216409 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":136,"threads_started":1}
I20260809 02:02:32.219033 3880 heartbeater.cc:347] Connected to a master server at 127.1.192.254:43253
I20260809 02:02:32.219310 3880 heartbeater.cc:464] Registering TS with master...
I20260809 02:02:32.219937 3880 heartbeater.cc:511] Master 127.1.192.254:43253 requested a full tablet report, sending...
I20260809 02:02:32.221686 3660 ts_manager.cc:194] Registered new tserver with Master: 3a1dc6be7d1640ba97604a9c62f597e9 (127.1.192.193:34563)
I20260809 02:02:32.222913 3660 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.192.193@KRBTEST.COM'} at 127.1.192.193:52355
WARNING: no policy specified for kudu/127.1.192.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.192.194@KRBTEST.COM" created.
I20260809 02:02:32.233741 3713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:32.221587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.193:47261 (local address 127.1.192.254:43253)
0809 02:02:32.221727 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:32.221731 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:32.225676 (+ 3945us) server_negotiation.cc:407] Connection header received
0809 02:02:32.225768 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:32.225774 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:32.225840 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:32.225952 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:32.226379 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:32.227279 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:32.228155 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:32.228541 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:32.228678 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:32.230004 (+ 1326us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:02:32.230012 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:02:32.230015 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:02:32.230040 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:02:32.231643 (+ 1603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:32.232248 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:32.232254 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:32.232261 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:32.232342 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:32.232689 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:32.232692 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:32.232693 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:32.232958 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:02:32.233083 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:32.233411 (+ 328us) server_negotiation.cc:299] Negotiation successful
0809 02:02:32.233600 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 02:02:32.233783 3887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:32.221537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:43253 (local address 127.1.192.193:47261)
0809 02:02:32.225558 (+ 4021us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:32.225579 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 02:02:32.225682 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:02:32.225954 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:02:32.225956 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:02:32.226150 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:02:32.226253 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:32.226259 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:32.227424 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:32.227427 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:32.228008 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:32.228028 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:32.228691 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:32.228694 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:32.228769 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:32.229372 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:02:32.229384 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:02:32.229849 (+ 465us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:02:32.231774 (+ 1925us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:32.231777 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:32.231778 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:32.232072 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:32.232435 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:32.232438 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:32.232439 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:32.232592 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:32.233076 (+ 484us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:02:32.233079 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:02:32.233173 (+ 94us) client_negotiation.cc:769] Sending connection context
0809 02:02:32.233312 (+ 139us) client_negotiation.cc:240] Negotiation successful
0809 02:02:32.233449 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3880,"thread_start_us":119,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.192.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.194.keytab.
Entry for principal kudu/127.1.192.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.194.keytab.
I20260809 02:02:32.254210 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786240941968412-1795-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786240941968412-1795-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786240941968412-1795-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786240941968412-1795-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.192.194:0
--local_ip_for_outbound_sockets=127.1.192.194
--webserver_interface=127.1.192.194
--webserver_port=0
--tserver_master_addrs=127.1.192.254:43253
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.194.keytab
--principal=kudu/127.1.192.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:36059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:02:32.371201 3889 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:02:32.371495 3889 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:02:32.371595 3889 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:02:32.375895 3889 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:02:32.375967 3889 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:02:32.376037 3889 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 02:02:32.376070 3889 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 02:02:32.376144 3889 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.192.194
I20260809 02:02:32.381954 3889 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:36059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786240941968412-1795-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786240941968412-1795-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.194.keytab
--principal=kudu/127.1.192.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.192.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786240941968412-1795-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.192.194
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.192.254:43253
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3889
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.192.194
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786240941968412-1795-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:02:32.383126 3889 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:02:32.384263 3889 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:02:32.394635 3895 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:02:32.394744 3889 server_base.cc:1034] running on GCE node
W20260809 02:02:32.394601 3897 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:02:32.394603 3894 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:02:32.395413 3889 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:02:32.396129 3889 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:02:32.397327 3889 hybrid_clock.cc:648] HybridClock initialized: now 1786240952397327 us; error 49 us; skew 500 ppm
Aug 09 02:02:32 dist-test-slave-ss4n krb5kdc[3613](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240952, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:02:32.400375 3889 init.cc:377] Logged in from keytab as kudu/127.1.192.194@KRBTEST.COM (short username kudu)
I20260809 02:02:32.401532 3889 webserver.cc:460] Webserver started at http://127.1.192.194:41739/ using document root <none> and password file <none>
I20260809 02:02:32.402137 3889 fs_manager.cc:359] Metadata directory not provided
I20260809 02:02:32.402246 3889 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:02:32.402467 3889 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:02:32.404590 3889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786240941968412-1795-0/minicluster-data/ts-1/data/instance:
uuid: "2ea47313d10a4705bf36329be4a87410"
format_stamp: "Formatted at 2026-08-09 02:02:32 on dist-test-slave-ss4n"
server_key: "80e66dcb0ca4e22313531c63410016a0"
server_key_iv: "dc3f13e5ecd24e3f943c295fcb14faa7"
server_key_version: "encryptionkey@0"
I20260809 02:02:32.405213 3889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786240941968412-1795-0/minicluster-data/ts-1/wal/instance:
uuid: "2ea47313d10a4705bf36329be4a87410"
format_stamp: "Formatted at 2026-08-09 02:02:32 on dist-test-slave-ss4n"
server_key: "80e66dcb0ca4e22313531c63410016a0"
server_key_iv: "dc3f13e5ecd24e3f943c295fcb14faa7"
server_key_version: "encryptionkey@0"
I20260809 02:02:32.408999 3889 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260809 02:02:32.411708 3905 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:32.412997 3889 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260809 02:02:32.413167 3889 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786240941968412-1795-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786240941968412-1795-0/minicluster-data/ts-1/wal
uuid: "2ea47313d10a4705bf36329be4a87410"
format_stamp: "Formatted at 2026-08-09 02:02:32 on dist-test-slave-ss4n"
server_key: "80e66dcb0ca4e22313531c63410016a0"
server_key_iv: "dc3f13e5ecd24e3f943c295fcb14faa7"
server_key_version: "encryptionkey@0"
I20260809 02:02:32.413293 3889 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786240941968412-1795-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786240941968412-1795-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786240941968412-1795-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:02:32.432161 3889 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:02:32.435667 3889 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:02:32.435896 3889 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:02:32.437556 3889 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:02:32.437628 3889 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:32.437688 3889 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:02:32.437724 3889 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 02:02:32 dist-test-slave-ss4n krb5kdc[3613](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240952, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.194@KRBTEST.COM for kudu/127.1.192.254@KRBTEST.COM
I20260809 02:02:32.449385 3889 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.194:38867
I20260809 02:02:32.449462 4027 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.194:38867 every 8 connection(s)
I20260809 02:02:32.450367 3889 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786240941968412-1795-0/minicluster-data/ts-1/data/info.pb
I20260809 02:02:32.452075 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 3889
I20260809 02:02:32.452205 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786240941968412-1795-0/minicluster-data/ts-1/wal/instance
I20260809 02:02:32.452476 1795 external_mini_cluster.cc:1428] Setting key aacc47e1268ec809397936496b2a3c8a
I20260809 02:02:32.454182 3713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:32.438499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.194:45257 (local address 127.1.192.254:43253)
0809 02:02:32.438697 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:32.438704 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:32.439657 (+ 953us) server_negotiation.cc:407] Connection header received
0809 02:02:32.441073 (+ 1416us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:32.441078 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:32.441156 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:32.441293 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:32.442830 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:32.443439 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:32.445511 (+ 2072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:32.445956 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:32.446160 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:32.449729 (+ 3569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:02:32.449733 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:02:32.449736 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:02:32.449778 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:02:32.451682 (+ 1904us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:32.452347 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:32.452351 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:32.452353 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:32.452453 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:32.452860 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:32.452865 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:32.452867 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:32.453144 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:02:32.453255 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:32.453852 (+ 597us) server_negotiation.cc:299] Negotiation successful
0809 02:02:32.453988 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 02:02:32.455021 3916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:32.438948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:43253 (local address 127.1.192.194:45257)
0809 02:02:32.439458 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:32.439524 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0809 02:02:32.440721 (+ 1197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:02:32.441540 (+ 819us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:02:32.441551 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:02:32.442054 (+ 503us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:02:32.442656 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:32.442672 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:32.443607 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:32.443610 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:32.445280 (+ 1670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:32.445295 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:32.446238 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:32.446243 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:32.446456 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:32.447419 (+ 963us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:02:32.447446 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:02:32.449536 (+ 2090us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:02:32.451813 (+ 2277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:32.451820 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:32.451836 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:32.452243 (+ 407us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:32.452573 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:32.452576 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:32.452579 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:32.452751 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:32.453250 (+ 499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:02:32.453257 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:02:32.453601 (+ 344us) client_negotiation.cc:769] Sending connection context
0809 02:02:32.453805 (+ 204us) client_negotiation.cc:240] Negotiation successful
0809 02:02:32.454157 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":118,"threads_started":1}
I20260809 02:02:32.456458 4028 heartbeater.cc:347] Connected to a master server at 127.1.192.254:43253
I20260809 02:02:32.456636 4028 heartbeater.cc:464] Registering TS with master...
I20260809 02:02:32.457288 4028 heartbeater.cc:511] Master 127.1.192.254:43253 requested a full tablet report, sending...
I20260809 02:02:32.458519 3658 ts_manager.cc:194] Registered new tserver with Master: 2ea47313d10a4705bf36329be4a87410 (127.1.192.194:38867)
I20260809 02:02:32.459365 3658 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.192.194@KRBTEST.COM'} at 127.1.192.194:45257
WARNING: no policy specified for kudu/127.1.192.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.192.195@KRBTEST.COM" created.
I20260809 02:02:32.468327 4036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:32.458786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:43253 (local address 127.1.192.194:45823)
0809 02:02:32.459072 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:32.459088 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 02:02:32.459177 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:02:32.459968 (+ 791us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:02:32.459974 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:02:32.460222 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:02:32.460356 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:32.460364 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:32.461753 (+ 1389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:32.461756 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:32.462313 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:32.462321 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:32.463038 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:32.463042 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:32.463118 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:32.463747 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:02:32.463759 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:02:32.464135 (+ 376us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:02:32.466349 (+ 2214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:32.466352 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:32.466354 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:32.466568 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:32.466907 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:32.466910 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:32.466912 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:32.467008 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:32.467686 (+ 678us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:02:32.467688 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:02:32.467794 (+ 106us) client_negotiation.cc:769] Sending connection context
0809 02:02:32.467902 (+ 108us) client_negotiation.cc:240] Negotiation successful
0809 02:02:32.468135 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":102,"threads_started":1}
I20260809 02:02:32.468444 3713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:32.458830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.194:45823 (local address 127.1.192.254:43253)
0809 02:02:32.459731 (+ 901us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:32.459737 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:32.459752 (+ 15us) server_negotiation.cc:407] Connection header received
0809 02:02:32.459787 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:32.459789 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:32.459832 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:32.459924 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:32.460499 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:32.461608 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:32.462438 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:32.462874 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:32.463006 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:32.464322 (+ 1316us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:02:32.464326 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:02:32.464329 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:02:32.464356 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:02:32.466214 (+ 1858us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:32.466684 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:32.466687 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:32.466689 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:32.466769 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:32.467269 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:32.467272 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:32.467273 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:32.467517 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:02:32.467672 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:32.468020 (+ 348us) server_negotiation.cc:299] Negotiation successful
0809 02:02:32.468206 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":802}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.192.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.195.keytab.
Entry for principal kudu/127.1.192.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.195.keytab.
I20260809 02:02:32.486703 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786240941968412-1795-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786240941968412-1795-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786240941968412-1795-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786240941968412-1795-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.192.195:0
--local_ip_for_outbound_sockets=127.1.192.195
--webserver_interface=127.1.192.195
--webserver_port=0
--tserver_master_addrs=127.1.192.254:43253
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.195.keytab
--principal=kudu/127.1.192.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:36059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:02:32.603050 4038 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:02:32.603343 4038 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:02:32.603394 4038 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:02:32.607463 4038 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:02:32.607537 4038 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:02:32.607579 4038 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 02:02:32.607594 4038 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 02:02:32.607627 4038 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.192.195
I20260809 02:02:32.613106 4038 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:36059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786240941968412-1795-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786240941968412-1795-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.195.keytab
--principal=kudu/127.1.192.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.192.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786240941968412-1795-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.192.195
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.192.254:43253
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4038
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.192.195
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786240941968412-1795-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:02:32.614307 4038 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:02:32.615397 4038 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:02:32.625864 4044 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:02:32.625870 4045 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:02:32.626086 4038 server_base.cc:1034] running on GCE node
W20260809 02:02:32.625864 4048 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:02:32.626608 4038 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:02:32.627277 4038 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:02:32.628445 4038 hybrid_clock.cc:648] HybridClock initialized: now 1786240952628426 us; error 32 us; skew 500 ppm
Aug 09 02:02:32 dist-test-slave-ss4n krb5kdc[3613](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240952, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:02:32.631889 4038 init.cc:377] Logged in from keytab as kudu/127.1.192.195@KRBTEST.COM (short username kudu)
I20260809 02:02:32.632908 4038 webserver.cc:460] Webserver started at http://127.1.192.195:34567/ using document root <none> and password file <none>
I20260809 02:02:32.633580 4038 fs_manager.cc:359] Metadata directory not provided
I20260809 02:02:32.633675 4038 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:02:32.633865 4038 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:02:32.636200 4038 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786240941968412-1795-0/minicluster-data/ts-2/data/instance:
uuid: "ed9f4412136e40cc82447ec2dbda71b9"
format_stamp: "Formatted at 2026-08-09 02:02:32 on dist-test-slave-ss4n"
server_key: "fc97a107e3fd3fc1bb8787bd7eae44db"
server_key_iv: "f4a559b5537632a4652895822d7c4336"
server_key_version: "encryptionkey@0"
I20260809 02:02:32.636782 4038 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786240941968412-1795-0/minicluster-data/ts-2/wal/instance:
uuid: "ed9f4412136e40cc82447ec2dbda71b9"
format_stamp: "Formatted at 2026-08-09 02:02:32 on dist-test-slave-ss4n"
server_key: "fc97a107e3fd3fc1bb8787bd7eae44db"
server_key_iv: "f4a559b5537632a4652895822d7c4336"
server_key_version: "encryptionkey@0"
I20260809 02:02:32.640865 4038 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 02:02:32.643685 4055 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:32.645133 4038 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 02:02:32.645253 4038 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786240941968412-1795-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786240941968412-1795-0/minicluster-data/ts-2/wal
uuid: "ed9f4412136e40cc82447ec2dbda71b9"
format_stamp: "Formatted at 2026-08-09 02:02:32 on dist-test-slave-ss4n"
server_key: "fc97a107e3fd3fc1bb8787bd7eae44db"
server_key_iv: "f4a559b5537632a4652895822d7c4336"
server_key_version: "encryptionkey@0"
I20260809 02:02:32.645361 4038 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786240941968412-1795-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786240941968412-1795-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786240941968412-1795-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:02:32.668403 4038 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:02:32.671619 4038 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:02:32.671788 4038 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:02:32.673288 4038 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:02:32.673341 4038 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:32.673383 4038 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:02:32.673445 4038 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 02:02:32 dist-test-slave-ss4n krb5kdc[3613](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240952, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.195@KRBTEST.COM for kudu/127.1.192.254@KRBTEST.COM
I20260809 02:02:32.685307 4038 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.195:43843
I20260809 02:02:32.685386 4190 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.195:43843 every 8 connection(s)
I20260809 02:02:32.686479 4038 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786240941968412-1795-0/minicluster-data/ts-2/data/info.pb
I20260809 02:02:32.687932 3713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:32.674231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.195:47571 (local address 127.1.192.254:43253)
0809 02:02:32.674376 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:32.674381 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:32.675311 (+ 930us) server_negotiation.cc:407] Connection header received
0809 02:02:32.676650 (+ 1339us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:32.676654 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:32.676739 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:32.676853 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:32.678495 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:32.679048 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:32.680214 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:32.680443 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:32.680557 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:32.683866 (+ 3309us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:02:32.683870 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:02:32.683877 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:02:32.683909 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:02:32.685713 (+ 1804us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:32.686239 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:32.686243 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:32.686245 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:32.686340 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:32.686776 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:32.686782 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:32.686786 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:32.687048 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:02:32.687156 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:32.687699 (+ 543us) server_negotiation.cc:299] Negotiation successful
0809 02:02:32.687807 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 02:02:32.688723 4066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:32.674679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:43253 (local address 127.1.192.195:47571)
0809 02:02:32.675138 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:32.675189 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 02:02:32.676296 (+ 1107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:02:32.677058 (+ 762us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:02:32.677068 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:02:32.677637 (+ 569us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:02:32.678336 (+ 699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:32.678351 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:32.679183 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:32.679187 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:32.680075 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:32.680083 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:32.680613 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:32.680617 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:32.680838 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:32.681655 (+ 817us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:02:32.681672 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:02:32.683680 (+ 2008us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:02:32.685846 (+ 2166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:32.685850 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:32.685861 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:32.686125 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:32.686441 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:32.686443 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:32.686445 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:32.686675 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:32.687217 (+ 542us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:02:32.687225 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:02:32.687490 (+ 265us) client_negotiation.cc:769] Sending connection context
0809 02:02:32.687647 (+ 157us) client_negotiation.cc:240] Negotiation successful
0809 02:02:32.687906 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":124,"threads_started":1}
I20260809 02:02:32.690161 4191 heartbeater.cc:347] Connected to a master server at 127.1.192.254:43253
I20260809 02:02:32.690338 4191 heartbeater.cc:464] Registering TS with master...
I20260809 02:02:32.690972 4191 heartbeater.cc:511] Master 127.1.192.254:43253 requested a full tablet report, sending...
I20260809 02:02:32.692183 3660 ts_manager.cc:194] Registered new tserver with Master: ed9f4412136e40cc82447ec2dbda71b9 (127.1.192.195:43843)
I20260809 02:02:32.692886 3660 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.192.195@KRBTEST.COM'} at 127.1.192.195:47571
I20260809 02:02:32.694777 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 4038
I20260809 02:02:32.694928 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786240941968412-1795-0/minicluster-data/ts-2/wal/instance
I20260809 02:02:32.695258 1795 external_mini_cluster.cc:1428] Setting key d6bd8b2dc9d715eb91adad9754846ef1
I20260809 02:02:32.697731 1795 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 02:02:32.701303 4197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:32.692463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:43253 (local address 127.1.192.195:41103)
0809 02:02:32.692772 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:32.692786 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 02:02:32.692866 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:02:32.693616 (+ 750us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:02:32.693619 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:02:32.693846 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:02:32.693953 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:32.693959 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:32.694942 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:32.694946 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:32.695677 (+ 731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:32.695687 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:32.696339 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:32.696342 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:32.696446 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:32.697100 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:02:32.697111 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:02:32.697573 (+ 462us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:02:32.699570 (+ 1997us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:32.699576 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:32.699581 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:32.699887 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:32.700278 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:32.700280 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:32.700281 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:32.700366 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:32.700839 (+ 473us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:02:32.700842 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:02:32.700937 (+ 95us) client_negotiation.cc:769] Sending connection context
0809 02:02:32.701027 (+ 90us) client_negotiation.cc:240] Negotiation successful
0809 02:02:32.701157 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":94,"threads_started":1}
I20260809 02:02:32.701508 3713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:32.693207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.195:41103 (local address 127.1.192.254:43253)
0809 02:02:32.693343 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:32.693346 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:32.693360 (+ 14us) server_negotiation.cc:407] Connection header received
0809 02:02:32.693393 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:32.693395 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:32.693429 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:32.693598 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:32.694092 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:32.694779 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:32.695826 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:32.696162 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:32.696286 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:32.697741 (+ 1455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:02:32.697744 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:02:32.697746 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:02:32.697769 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:02:32.699447 (+ 1678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:32.700041 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:32.700046 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:32.700049 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:32.700165 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:32.700460 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:32.700465 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:32.700467 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:32.700712 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:02:32.700828 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:32.701150 (+ 322us) server_negotiation.cc:299] Negotiation successful
0809 02:02:32.701295 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
Aug 09 02:02:32 dist-test-slave-ss4n krb5kdc[3613](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240952, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 02:02:32 dist-test-slave-ss4n krb5kdc[3613](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240952, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.192.254@KRBTEST.COM
I20260809 02:02:32.726673 3713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:32.718290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36318 (local address 127.1.192.254:43253)
0809 02:02:32.718445 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:32.718454 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:32.718467 (+ 13us) server_negotiation.cc:407] Connection header received
0809 02:02:32.718507 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:32.718509 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:32.718598 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:32.718696 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:32.719513 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:32.720063 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:32.720919 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:32.721154 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:32.721240 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:32.723733 (+ 2493us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:02:32.723736 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:02:32.723738 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:02:32.723763 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:02:32.725275 (+ 1512us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:32.725722 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:32.725724 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:32.725726 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:32.725801 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:32.726018 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:32.726020 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:32.726021 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:32.726181 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:02:32.726279 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:32.726445 (+ 166us) server_negotiation.cc:299] Negotiation successful
0809 02:02:32.726548 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 02:02:32.730194 3660 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36318:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 02:02:32.734069 3660 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:02:32.749790 4215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:32.744235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48422 (local address 127.1.192.193:34563)
0809 02:02:32.744634 (+ 399us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:32.744640 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:32.745085 (+ 445us) server_negotiation.cc:407] Connection header received
0809 02:02:32.745170 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:32.745175 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:32.745349 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:32.745503 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:02:32.745847 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:32.747185 (+ 1338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:32.748339 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:32.749011 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:32.749136 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:32.749290 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:32.749508 (+ 218us) server_negotiation.cc:299] Negotiation successful
0809 02:02:32.749623 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":302,"thread_start_us":105,"threads_started":1}
I20260809 02:02:32.750157 4209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:32.744055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.193:34563 (local address 127.0.0.1:48422)
0809 02:02:32.744980 (+ 925us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:32.744994 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 02:02:32.745078 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:02:32.745562 (+ 484us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:02:32.745568 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:02:32.745590 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:02:32.745689 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:32.745695 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:32.747305 (+ 1610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:32.747308 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:32.748203 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:32.748217 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:32.749136 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:32.749139 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:32.749254 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:32.749291 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 02:02:32.749557 (+ 266us) client_negotiation.cc:240] Negotiation successful
0809 02:02:32.750009 (+ 452us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":814,"thread_start_us":105,"threads_started":1}
I20260809 02:02:32.750486 4210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:32.743224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59128 (local address 127.1.192.195:43843)
0809 02:02:32.743551 (+ 327us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:32.743589 (+ 38us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:32.744410 (+ 821us) server_negotiation.cc:407] Connection header received
0809 02:02:32.744495 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:32.744502 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:32.744740 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:32.744908 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:02:32.745533 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:32.746800 (+ 1267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:32.749130 (+ 2330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:32.749835 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:32.749966 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:32.750097 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:32.750217 (+ 120us) server_negotiation.cc:299] Negotiation successful
0809 02:02:32.750306 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":217,"thread_start_us":122,"threads_started":1}
I20260809 02:02:32.750928 4212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:32.743055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.195:43843 (local address 127.0.0.1:59128)
0809 02:02:32.744049 (+ 994us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:32.744096 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 02:02:32.744253 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:02:32.745095 (+ 842us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:02:32.745102 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:02:32.745130 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:02:32.745349 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:32.745357 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:32.746914 (+ 1557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:32.746917 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:32.748977 (+ 2060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:32.748989 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:32.749968 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:32.749971 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:32.750072 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:32.750094 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 02:02:32.750187 (+ 93us) client_negotiation.cc:240] Negotiation successful
0809 02:02:32.750770 (+ 583us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":852,"mutex_wait_us":906,"thread_start_us":118,"threads_started":1}
I20260809 02:02:32.751163 4211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:32.743299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54768 (local address 127.1.192.194:38867)
0809 02:02:32.743787 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:32.743802 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:32.744396 (+ 594us) server_negotiation.cc:407] Connection header received
0809 02:02:32.744619 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:32.744626 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:32.744839 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:32.744981 (+ 142us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:02:32.745452 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:32.746598 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:32.748198 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:32.749011 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:32.749136 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:32.749289 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:32.750883 (+ 1594us) server_negotiation.cc:299] Negotiation successful
0809 02:02:32.750995 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":384,"thread_start_us":114,"threads_started":1}
I20260809 02:02:32.751359 4214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:32.743064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.194:38867 (local address 127.0.0.1:54768)
0809 02:02:32.744294 (+ 1230us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:32.744308 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 02:02:32.744467 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:02:32.745171 (+ 704us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:02:32.745176 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:02:32.745196 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:02:32.745336 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:32.745341 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:32.746710 (+ 1369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:32.746714 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:32.748058 (+ 1344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:32.748069 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:32.749751 (+ 1682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:32.749754 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:32.750289 (+ 535us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:32.750749 (+ 460us) client_negotiation.cc:769] Sending connection context
0809 02:02:32.750832 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 02:02:32.750903 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1160,"mutex_wait_us":1060,"thread_start_us":113,"threads_started":1}
I20260809 02:02:32.753584 4113 tablet_service.cc:1467] Processing CreateTablet for tablet f130ab67eb1d4a8884ab0d0620709b5c (DEFAULT_TABLE table=test-table [id=9c9b7bdade2d4970b57802afd1560fac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:02:32.753582 3811 tablet_service.cc:1467] Processing CreateTablet for tablet f130ab67eb1d4a8884ab0d0620709b5c (DEFAULT_TABLE table=test-table [id=9c9b7bdade2d4970b57802afd1560fac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:02:32.754195 3958 tablet_service.cc:1467] Processing CreateTablet for tablet f130ab67eb1d4a8884ab0d0620709b5c (DEFAULT_TABLE table=test-table [id=9c9b7bdade2d4970b57802afd1560fac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:02:32.754598 4113 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f130ab67eb1d4a8884ab0d0620709b5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:02:32.754628 3811 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f130ab67eb1d4a8884ab0d0620709b5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:02:32.755100 3958 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f130ab67eb1d4a8884ab0d0620709b5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:02:32.761729 4216 tablet_bootstrap.cc:492] T f130ab67eb1d4a8884ab0d0620709b5c P ed9f4412136e40cc82447ec2dbda71b9: Bootstrap starting.
I20260809 02:02:32.762897 4218 tablet_bootstrap.cc:492] T f130ab67eb1d4a8884ab0d0620709b5c P 3a1dc6be7d1640ba97604a9c62f597e9: Bootstrap starting.
I20260809 02:02:32.764356 4216 tablet_bootstrap.cc:654] T f130ab67eb1d4a8884ab0d0620709b5c P ed9f4412136e40cc82447ec2dbda71b9: Neither blocks nor log segments found. Creating new log.
I20260809 02:02:32.764603 4218 tablet_bootstrap.cc:654] T f130ab67eb1d4a8884ab0d0620709b5c P 3a1dc6be7d1640ba97604a9c62f597e9: Neither blocks nor log segments found. Creating new log.
I20260809 02:02:32.765134 4216 log.cc:824] T f130ab67eb1d4a8884ab0d0620709b5c P ed9f4412136e40cc82447ec2dbda71b9: Log is configured to *not* fsync() on all Append() calls
I20260809 02:02:32.765311 4218 log.cc:824] T f130ab67eb1d4a8884ab0d0620709b5c P 3a1dc6be7d1640ba97604a9c62f597e9: Log is configured to *not* fsync() on all Append() calls
I20260809 02:02:32.765854 4219 tablet_bootstrap.cc:492] T f130ab67eb1d4a8884ab0d0620709b5c P 2ea47313d10a4705bf36329be4a87410: Bootstrap starting.
I20260809 02:02:32.767356 4216 tablet_bootstrap.cc:492] T f130ab67eb1d4a8884ab0d0620709b5c P ed9f4412136e40cc82447ec2dbda71b9: No bootstrap required, opened a new log
I20260809 02:02:32.767356 4218 tablet_bootstrap.cc:492] T f130ab67eb1d4a8884ab0d0620709b5c P 3a1dc6be7d1640ba97604a9c62f597e9: No bootstrap required, opened a new log
I20260809 02:02:32.767570 4218 ts_tablet_manager.cc:1397] T f130ab67eb1d4a8884ab0d0620709b5c P 3a1dc6be7d1640ba97604a9c62f597e9: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260809 02:02:32.767558 4216 ts_tablet_manager.cc:1397] T f130ab67eb1d4a8884ab0d0620709b5c P ed9f4412136e40cc82447ec2dbda71b9: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260809 02:02:32.768039 4219 tablet_bootstrap.cc:654] T f130ab67eb1d4a8884ab0d0620709b5c P 2ea47313d10a4705bf36329be4a87410: Neither blocks nor log segments found. Creating new log.
I20260809 02:02:32.768738 4219 log.cc:824] T f130ab67eb1d4a8884ab0d0620709b5c P 2ea47313d10a4705bf36329be4a87410: Log is configured to *not* fsync() on all Append() calls
I20260809 02:02:32.770533 4219 tablet_bootstrap.cc:492] T f130ab67eb1d4a8884ab0d0620709b5c P 2ea47313d10a4705bf36329be4a87410: No bootstrap required, opened a new log
I20260809 02:02:32.770767 4219 ts_tablet_manager.cc:1397] T f130ab67eb1d4a8884ab0d0620709b5c P 2ea47313d10a4705bf36329be4a87410: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260809 02:02:32.771724 4218 raft_consensus.cc:348] T f130ab67eb1d4a8884ab0d0620709b5c P 3a1dc6be7d1640ba97604a9c62f597e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed9f4412136e40cc82447ec2dbda71b9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 43843 } } peers { permanent_uuid: "2ea47313d10a4705bf36329be4a87410" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38867 } } peers { permanent_uuid: "3a1dc6be7d1640ba97604a9c62f597e9" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 34563 } }
I20260809 02:02:32.771724 4216 raft_consensus.cc:348] T f130ab67eb1d4a8884ab0d0620709b5c P ed9f4412136e40cc82447ec2dbda71b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed9f4412136e40cc82447ec2dbda71b9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 43843 } } peers { permanent_uuid: "2ea47313d10a4705bf36329be4a87410" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38867 } } peers { permanent_uuid: "3a1dc6be7d1640ba97604a9c62f597e9" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 34563 } }
I20260809 02:02:32.772006 4218 raft_consensus.cc:374] T f130ab67eb1d4a8884ab0d0620709b5c P 3a1dc6be7d1640ba97604a9c62f597e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:02:32.772006 4216 raft_consensus.cc:374] T f130ab67eb1d4a8884ab0d0620709b5c P ed9f4412136e40cc82447ec2dbda71b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:02:32.772065 4218 raft_consensus.cc:729] T f130ab67eb1d4a8884ab0d0620709b5c P 3a1dc6be7d1640ba97604a9c62f597e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a1dc6be7d1640ba97604a9c62f597e9, State: Initialized, Role: FOLLOWER
I20260809 02:02:32.772075 4216 raft_consensus.cc:729] T f130ab67eb1d4a8884ab0d0620709b5c P ed9f4412136e40cc82447ec2dbda71b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed9f4412136e40cc82447ec2dbda71b9, State: Initialized, Role: FOLLOWER
I20260809 02:02:32.772538 4216 consensus_queue.cc:260] T f130ab67eb1d4a8884ab0d0620709b5c P ed9f4412136e40cc82447ec2dbda71b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed9f4412136e40cc82447ec2dbda71b9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 43843 } } peers { permanent_uuid: "2ea47313d10a4705bf36329be4a87410" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38867 } } peers { permanent_uuid: "3a1dc6be7d1640ba97604a9c62f597e9" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 34563 } }
I20260809 02:02:32.772539 4218 consensus_queue.cc:260] T f130ab67eb1d4a8884ab0d0620709b5c P 3a1dc6be7d1640ba97604a9c62f597e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed9f4412136e40cc82447ec2dbda71b9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 43843 } } peers { permanent_uuid: "2ea47313d10a4705bf36329be4a87410" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38867 } } peers { permanent_uuid: "3a1dc6be7d1640ba97604a9c62f597e9" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 34563 } }
I20260809 02:02:32.773273 3880 heartbeater.cc:503] Master 127.1.192.254:43253 was elected leader, sending a full tablet report...
I20260809 02:02:32.773590 4218 ts_tablet_manager.cc:1428] T f130ab67eb1d4a8884ab0d0620709b5c P 3a1dc6be7d1640ba97604a9c62f597e9: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260809 02:02:32.773586 4216 ts_tablet_manager.cc:1428] T f130ab67eb1d4a8884ab0d0620709b5c P ed9f4412136e40cc82447ec2dbda71b9: Time spent starting tablet: real 0.006s user 0.004s sys 0.003s
I20260809 02:02:32.773916 4191 heartbeater.cc:503] Master 127.1.192.254:43253 was elected leader, sending a full tablet report...
I20260809 02:02:32.774305 4219 raft_consensus.cc:348] T f130ab67eb1d4a8884ab0d0620709b5c P 2ea47313d10a4705bf36329be4a87410 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed9f4412136e40cc82447ec2dbda71b9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 43843 } } peers { permanent_uuid: "2ea47313d10a4705bf36329be4a87410" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38867 } } peers { permanent_uuid: "3a1dc6be7d1640ba97604a9c62f597e9" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 34563 } }
I20260809 02:02:32.774596 4219 raft_consensus.cc:374] T f130ab67eb1d4a8884ab0d0620709b5c P 2ea47313d10a4705bf36329be4a87410 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:02:32.774827 4219 raft_consensus.cc:729] T f130ab67eb1d4a8884ab0d0620709b5c P 2ea47313d10a4705bf36329be4a87410 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ea47313d10a4705bf36329be4a87410, State: Initialized, Role: FOLLOWER
I20260809 02:02:32.775334 4219 consensus_queue.cc:260] T f130ab67eb1d4a8884ab0d0620709b5c P 2ea47313d10a4705bf36329be4a87410 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed9f4412136e40cc82447ec2dbda71b9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 43843 } } peers { permanent_uuid: "2ea47313d10a4705bf36329be4a87410" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38867 } } peers { permanent_uuid: "3a1dc6be7d1640ba97604a9c62f597e9" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 34563 } }
I20260809 02:02:32.776479 4219 ts_tablet_manager.cc:1428] T f130ab67eb1d4a8884ab0d0620709b5c P 2ea47313d10a4705bf36329be4a87410: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260809 02:02:32.776839 4028 heartbeater.cc:503] Master 127.1.192.254:43253 was elected leader, sending a full tablet report...
I20260809 02:02:32.811105 4223 raft_consensus.cc:482] T f130ab67eb1d4a8884ab0d0620709b5c P 3a1dc6be7d1640ba97604a9c62f597e9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:02:32.811329 4223 raft_consensus.cc:504] T f130ab67eb1d4a8884ab0d0620709b5c P 3a1dc6be7d1640ba97604a9c62f597e9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed9f4412136e40cc82447ec2dbda71b9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 43843 } } peers { permanent_uuid: "2ea47313d10a4705bf36329be4a87410" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38867 } } peers { permanent_uuid: "3a1dc6be7d1640ba97604a9c62f597e9" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 34563 } }
I20260809 02:02:32.812426 4223 leader_election.cc:288] T f130ab67eb1d4a8884ab0d0620709b5c P 3a1dc6be7d1640ba97604a9c62f597e9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ed9f4412136e40cc82447ec2dbda71b9 (127.1.192.195:43843), 2ea47313d10a4705bf36329be4a87410 (127.1.192.194:38867)
I20260809 02:02:32.816732 4210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:32.812869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.193:50719 (local address 127.1.192.195:43843)
0809 02:02:32.813104 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:32.813109 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:32.813167 (+ 58us) server_negotiation.cc:407] Connection header received
0809 02:02:32.813315 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:32.813319 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:32.813367 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:32.813447 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:02:32.813845 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:32.814423 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:32.815639 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:32.816060 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:32.816210 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:32.816272 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:32.816513 (+ 241us) server_negotiation.cc:299] Negotiation successful
0809 02:02:32.816591 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148}
I20260809 02:02:32.816756 4211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:32.812839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.193:33037 (local address 127.1.192.194:38867)
0809 02:02:32.812981 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:32.812986 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:32.813336 (+ 350us) server_negotiation.cc:407] Connection header received
0809 02:02:32.813512 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:32.813515 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:32.813564 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:32.813643 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:02:32.813919 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:32.814422 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:32.815581 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:32.816072 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:32.816212 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:32.816266 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:32.816552 (+ 286us) server_negotiation.cc:299] Negotiation successful
0809 02:02:32.816626 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 02:02:32.816797 4226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:32.812702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.195:43843 (local address 127.1.192.193:50719)
0809 02:02:32.813084 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:32.813100 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 02:02:32.813201 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:02:32.813506 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:02:32.813509 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:02:32.813527 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:02:32.813721 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:32.813730 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:32.814544 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:32.814548 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:32.815493 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:32.815502 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:32.816190 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:32.816194 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:32.816375 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:32.816404 (+ 29us) client_negotiation.cc:769] Sending connection context
0809 02:02:32.816495 (+ 91us) client_negotiation.cc:240] Negotiation successful
0809 02:02:32.816606 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304,"mutex_wait_us":10,"thread_start_us":153,"threads_started":1}
I20260809 02:02:32.817337 4227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:32.812702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.194:38867 (local address 127.1.192.193:33037)
0809 02:02:32.813188 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:32.813202 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 02:02:32.813347 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:02:32.813669 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:02:32.813671 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:02:32.813686 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:02:32.813808 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:32.813813 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:32.814539 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:32.814545 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:32.815459 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:32.815469 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:32.816218 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:32.816221 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:32.816377 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:32.816406 (+ 29us) client_negotiation.cc:769] Sending connection context
0809 02:02:32.816501 (+ 95us) client_negotiation.cc:240] Negotiation successful
0809 02:02:32.816596 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":420,"mutex_wait_us":179,"thread_start_us":118,"threads_started":1}
I20260809 02:02:32.817404 4136 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f130ab67eb1d4a8884ab0d0620709b5c" candidate_uuid: "3a1dc6be7d1640ba97604a9c62f597e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed9f4412136e40cc82447ec2dbda71b9" is_pre_election: true
I20260809 02:02:32.817759 4136 raft_consensus.cc:2454] T f130ab67eb1d4a8884ab0d0620709b5c P ed9f4412136e40cc82447ec2dbda71b9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a1dc6be7d1640ba97604a9c62f597e9 in term 0.
I20260809 02:02:32.818303 3756 leader_election.cc:302] T f130ab67eb1d4a8884ab0d0620709b5c P 3a1dc6be7d1640ba97604a9c62f597e9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a1dc6be7d1640ba97604a9c62f597e9, ed9f4412136e40cc82447ec2dbda71b9; no voters:
I20260809 02:02:32.818542 3979 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f130ab67eb1d4a8884ab0d0620709b5c" candidate_uuid: "3a1dc6be7d1640ba97604a9c62f597e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ea47313d10a4705bf36329be4a87410" is_pre_election: true
I20260809 02:02:32.818632 4223 raft_consensus.cc:2781] T f130ab67eb1d4a8884ab0d0620709b5c P 3a1dc6be7d1640ba97604a9c62f597e9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:02:32.818708 4223 raft_consensus.cc:482] T f130ab67eb1d4a8884ab0d0620709b5c P 3a1dc6be7d1640ba97604a9c62f597e9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:02:32.818771 4223 raft_consensus.cc:3037] T f130ab67eb1d4a8884ab0d0620709b5c P 3a1dc6be7d1640ba97604a9c62f597e9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:02:32.818883 3979 raft_consensus.cc:2454] T f130ab67eb1d4a8884ab0d0620709b5c P 2ea47313d10a4705bf36329be4a87410 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a1dc6be7d1640ba97604a9c62f597e9 in term 0.
I20260809 02:02:32.820107 4223 raft_consensus.cc:504] T f130ab67eb1d4a8884ab0d0620709b5c P 3a1dc6be7d1640ba97604a9c62f597e9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed9f4412136e40cc82447ec2dbda71b9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 43843 } } peers { permanent_uuid: "2ea47313d10a4705bf36329be4a87410" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38867 } } peers { permanent_uuid: "3a1dc6be7d1640ba97604a9c62f597e9" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 34563 } }
I20260809 02:02:32.820498 4223 leader_election.cc:288] T f130ab67eb1d4a8884ab0d0620709b5c P 3a1dc6be7d1640ba97604a9c62f597e9 [CANDIDATE]: Term 1 election: Requested vote from peers ed9f4412136e40cc82447ec2dbda71b9 (127.1.192.195:43843), 2ea47313d10a4705bf36329be4a87410 (127.1.192.194:38867)
I20260809 02:02:32.820870 3979 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f130ab67eb1d4a8884ab0d0620709b5c" candidate_uuid: "3a1dc6be7d1640ba97604a9c62f597e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ea47313d10a4705bf36329be4a87410"
I20260809 02:02:32.820894 4136 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f130ab67eb1d4a8884ab0d0620709b5c" candidate_uuid: "3a1dc6be7d1640ba97604a9c62f597e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed9f4412136e40cc82447ec2dbda71b9"
I20260809 02:02:32.820998 4136 raft_consensus.cc:3037] T f130ab67eb1d4a8884ab0d0620709b5c P ed9f4412136e40cc82447ec2dbda71b9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:02:32.821007 3979 raft_consensus.cc:3037] T f130ab67eb1d4a8884ab0d0620709b5c P 2ea47313d10a4705bf36329be4a87410 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:02:32.822211 3979 raft_consensus.cc:2454] T f130ab67eb1d4a8884ab0d0620709b5c P 2ea47313d10a4705bf36329be4a87410 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a1dc6be7d1640ba97604a9c62f597e9 in term 1.
I20260809 02:02:32.822281 4136 raft_consensus.cc:2454] T f130ab67eb1d4a8884ab0d0620709b5c P ed9f4412136e40cc82447ec2dbda71b9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a1dc6be7d1640ba97604a9c62f597e9 in term 1.
I20260809 02:02:32.822628 3758 leader_election.cc:302] T f130ab67eb1d4a8884ab0d0620709b5c P 3a1dc6be7d1640ba97604a9c62f597e9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ea47313d10a4705bf36329be4a87410, 3a1dc6be7d1640ba97604a9c62f597e9; no voters:
I20260809 02:02:32.822834 4223 raft_consensus.cc:2781] T f130ab67eb1d4a8884ab0d0620709b5c P 3a1dc6be7d1640ba97604a9c62f597e9 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:02:32.823083 4223 raft_consensus.cc:686] T f130ab67eb1d4a8884ab0d0620709b5c P 3a1dc6be7d1640ba97604a9c62f597e9 [term 1 LEADER]: Becoming Leader. State: Replica: 3a1dc6be7d1640ba97604a9c62f597e9, State: Running, Role: LEADER
I20260809 02:02:32.823422 4223 consensus_queue.cc:237] T f130ab67eb1d4a8884ab0d0620709b5c P 3a1dc6be7d1640ba97604a9c62f597e9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed9f4412136e40cc82447ec2dbda71b9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 43843 } } peers { permanent_uuid: "2ea47313d10a4705bf36329be4a87410" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38867 } } peers { permanent_uuid: "3a1dc6be7d1640ba97604a9c62f597e9" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 34563 } }
I20260809 02:02:32.826890 3659 catalog_manager.cc:5495] T f130ab67eb1d4a8884ab0d0620709b5c P 3a1dc6be7d1640ba97604a9c62f597e9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3a1dc6be7d1640ba97604a9c62f597e9 (127.1.192.193). New cstate: current_term: 1 leader_uuid: "3a1dc6be7d1640ba97604a9c62f597e9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed9f4412136e40cc82447ec2dbda71b9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 43843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ea47313d10a4705bf36329be4a87410" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38867 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a1dc6be7d1640ba97604a9c62f597e9" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 34563 } health_report { overall_health: HEALTHY } } }
Aug 09 02:02:32 dist-test-slave-ss4n krb5kdc[3613](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240951, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.254@KRBTEST.COM for kudu/127.1.192.254@KRBTEST.COM
I20260809 02:02:32.876379 4236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:32.867404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:43253 (local address 127.0.0.1:36330)
0809 02:02:32.867810 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:32.867822 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:02:32.867964 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:02:32.868240 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:02:32.868243 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:02:32.868685 (+ 442us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:02:32.868786 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:32.868793 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:32.869639 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:32.869642 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:32.870251 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:32.870261 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:32.870717 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:32.870720 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:32.870802 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:32.871390 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:02:32.871403 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:02:32.873000 (+ 1597us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:02:32.874634 (+ 1634us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:32.874643 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:32.874648 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:32.874880 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:32.875206 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:32.875209 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:32.875210 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:32.875294 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:32.875750 (+ 456us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:02:32.875755 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:02:32.875882 (+ 127us) client_negotiation.cc:769] Sending connection context
0809 02:02:32.875975 (+ 93us) client_negotiation.cc:240] Negotiation successful
0809 02:02:32.876198 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":132,"threads_started":1}
I20260809 02:02:32.876379 3713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:32.867522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36330 (local address 127.1.192.254:43253)
0809 02:02:32.867730 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:32.867735 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:32.867905 (+ 170us) server_negotiation.cc:407] Connection header received
0809 02:02:32.868085 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:32.868088 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:32.868132 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:32.868222 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:32.868913 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:32.869441 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:32.870385 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:32.870601 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:32.870713 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:32.873137 (+ 2424us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:02:32.873141 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:02:32.873143 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:02:32.873171 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:02:32.874525 (+ 1354us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:32.875008 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:32.875012 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:32.875013 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:32.875113 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:32.875399 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:32.875401 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:32.875403 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:32.875644 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:02:32.875745 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:32.876043 (+ 298us) server_negotiation.cc:299] Negotiation successful
0809 02:02:32.876248 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":90,"server-negotiator.queue_time_us":63}
I20260809 02:02:32.880674 3659 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.1.192.254@KRBTEST.COM'} at 127.0.0.1:36330:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 02:02:32.881422 3659 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:02:32.886451 3958 tablet_service.cc:1467] Processing CreateTablet for tablet 1c71f361364a4e2ea1872308a3e10b7d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=1fe6a0d799924fe2b070f8503728e04c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 02:02:32.886260 3811 tablet_service.cc:1467] Processing CreateTablet for tablet 1c71f361364a4e2ea1872308a3e10b7d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=1fe6a0d799924fe2b070f8503728e04c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 02:02:32.886487 4113 tablet_service.cc:1467] Processing CreateTablet for tablet 1c71f361364a4e2ea1872308a3e10b7d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=1fe6a0d799924fe2b070f8503728e04c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 02:02:32.887068 3958 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c71f361364a4e2ea1872308a3e10b7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:02:32.887068 3811 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c71f361364a4e2ea1872308a3e10b7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:02:32.887205 4113 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c71f361364a4e2ea1872308a3e10b7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:02:32.890048 4216 tablet_bootstrap.cc:492] T 1c71f361364a4e2ea1872308a3e10b7d P ed9f4412136e40cc82447ec2dbda71b9: Bootstrap starting.
I20260809 02:02:32.890081 4219 tablet_bootstrap.cc:492] T 1c71f361364a4e2ea1872308a3e10b7d P 2ea47313d10a4705bf36329be4a87410: Bootstrap starting.
I20260809 02:02:32.890996 4219 tablet_bootstrap.cc:654] T 1c71f361364a4e2ea1872308a3e10b7d P 2ea47313d10a4705bf36329be4a87410: Neither blocks nor log segments found. Creating new log.
I20260809 02:02:32.891055 4216 tablet_bootstrap.cc:654] T 1c71f361364a4e2ea1872308a3e10b7d P ed9f4412136e40cc82447ec2dbda71b9: Neither blocks nor log segments found. Creating new log.
I20260809 02:02:32.892009 4218 tablet_bootstrap.cc:492] T 1c71f361364a4e2ea1872308a3e10b7d P 3a1dc6be7d1640ba97604a9c62f597e9: Bootstrap starting.
I20260809 02:02:32.892386 4219 tablet_bootstrap.cc:492] T 1c71f361364a4e2ea1872308a3e10b7d P 2ea47313d10a4705bf36329be4a87410: No bootstrap required, opened a new log
I20260809 02:02:32.892493 4219 ts_tablet_manager.cc:1397] T 1c71f361364a4e2ea1872308a3e10b7d P 2ea47313d10a4705bf36329be4a87410: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 02:02:32.892917 4216 tablet_bootstrap.cc:492] T 1c71f361364a4e2ea1872308a3e10b7d P ed9f4412136e40cc82447ec2dbda71b9: No bootstrap required, opened a new log
I20260809 02:02:32.893002 4216 ts_tablet_manager.cc:1397] T 1c71f361364a4e2ea1872308a3e10b7d P ed9f4412136e40cc82447ec2dbda71b9: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 02:02:32.893131 4218 tablet_bootstrap.cc:654] T 1c71f361364a4e2ea1872308a3e10b7d P 3a1dc6be7d1640ba97604a9c62f597e9: Neither blocks nor log segments found. Creating new log.
I20260809 02:02:32.893047 4219 raft_consensus.cc:348] T 1c71f361364a4e2ea1872308a3e10b7d P 2ea47313d10a4705bf36329be4a87410 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1dc6be7d1640ba97604a9c62f597e9" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 34563 } } peers { permanent_uuid: "2ea47313d10a4705bf36329be4a87410" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38867 } } peers { permanent_uuid: "ed9f4412136e40cc82447ec2dbda71b9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 43843 } }
I20260809 02:02:32.893182 4219 raft_consensus.cc:374] T 1c71f361364a4e2ea1872308a3e10b7d P 2ea47313d10a4705bf36329be4a87410 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:02:32.893219 4219 raft_consensus.cc:729] T 1c71f361364a4e2ea1872308a3e10b7d P 2ea47313d10a4705bf36329be4a87410 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ea47313d10a4705bf36329be4a87410, State: Initialized, Role: FOLLOWER
I20260809 02:02:32.893360 4219 consensus_queue.cc:260] T 1c71f361364a4e2ea1872308a3e10b7d P 2ea47313d10a4705bf36329be4a87410 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1dc6be7d1640ba97604a9c62f597e9" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 34563 } } peers { permanent_uuid: "2ea47313d10a4705bf36329be4a87410" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38867 } } peers { permanent_uuid: "ed9f4412136e40cc82447ec2dbda71b9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 43843 } }
I20260809 02:02:32.893599 4216 raft_consensus.cc:348] T 1c71f361364a4e2ea1872308a3e10b7d P ed9f4412136e40cc82447ec2dbda71b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1dc6be7d1640ba97604a9c62f597e9" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 34563 } } peers { permanent_uuid: "2ea47313d10a4705bf36329be4a87410" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38867 } } peers { permanent_uuid: "ed9f4412136e40cc82447ec2dbda71b9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 43843 } }
I20260809 02:02:32.893731 4216 raft_consensus.cc:374] T 1c71f361364a4e2ea1872308a3e10b7d P ed9f4412136e40cc82447ec2dbda71b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:02:32.893769 4216 raft_consensus.cc:729] T 1c71f361364a4e2ea1872308a3e10b7d P ed9f4412136e40cc82447ec2dbda71b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed9f4412136e40cc82447ec2dbda71b9, State: Initialized, Role: FOLLOWER
I20260809 02:02:32.893714 4225 tablet_replica.cc:439] T 1c71f361364a4e2ea1872308a3e10b7d P 2ea47313d10a4705bf36329be4a87410: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1dc6be7d1640ba97604a9c62f597e9" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 34563 } } peers { permanent_uuid: "2ea47313d10a4705bf36329be4a87410" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38867 } } peers { permanent_uuid: "ed9f4412136e40cc82447ec2dbda71b9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 43843 } } }
I20260809 02:02:32.893874 4225 tablet_replica.cc:442] T 1c71f361364a4e2ea1872308a3e10b7d P 2ea47313d10a4705bf36329be4a87410: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 02:02:32.893910 4216 consensus_queue.cc:260] T 1c71f361364a4e2ea1872308a3e10b7d P ed9f4412136e40cc82447ec2dbda71b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1dc6be7d1640ba97604a9c62f597e9" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 34563 } } peers { permanent_uuid: "2ea47313d10a4705bf36329be4a87410" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38867 } } peers { permanent_uuid: "ed9f4412136e40cc82447ec2dbda71b9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 43843 } }
I20260809 02:02:32.894212 4216 ts_tablet_manager.cc:1428] T 1c71f361364a4e2ea1872308a3e10b7d P ed9f4412136e40cc82447ec2dbda71b9: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 02:02:32.894188 4224 tablet_replica.cc:439] T 1c71f361364a4e2ea1872308a3e10b7d P ed9f4412136e40cc82447ec2dbda71b9: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1dc6be7d1640ba97604a9c62f597e9" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 34563 } } peers { permanent_uuid: "2ea47313d10a4705bf36329be4a87410" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38867 } } peers { permanent_uuid: "ed9f4412136e40cc82447ec2dbda71b9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 43843 } } }
I20260809 02:02:32.894359 4224 tablet_replica.cc:442] T 1c71f361364a4e2ea1872308a3e10b7d P ed9f4412136e40cc82447ec2dbda71b9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 02:02:32.894708 4219 ts_tablet_manager.cc:1428] T 1c71f361364a4e2ea1872308a3e10b7d P 2ea47313d10a4705bf36329be4a87410: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260809 02:02:32.895591 4218 tablet_bootstrap.cc:492] T 1c71f361364a4e2ea1872308a3e10b7d P 3a1dc6be7d1640ba97604a9c62f597e9: No bootstrap required, opened a new log
I20260809 02:02:32.895696 4218 ts_tablet_manager.cc:1397] T 1c71f361364a4e2ea1872308a3e10b7d P 3a1dc6be7d1640ba97604a9c62f597e9: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260809 02:02:32.896258 4218 raft_consensus.cc:348] T 1c71f361364a4e2ea1872308a3e10b7d P 3a1dc6be7d1640ba97604a9c62f597e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1dc6be7d1640ba97604a9c62f597e9" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 34563 } } peers { permanent_uuid: "2ea47313d10a4705bf36329be4a87410" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38867 } } peers { permanent_uuid: "ed9f4412136e40cc82447ec2dbda71b9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 43843 } }
I20260809 02:02:32.896370 4218 raft_consensus.cc:374] T 1c71f361364a4e2ea1872308a3e10b7d P 3a1dc6be7d1640ba97604a9c62f597e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:02:32.896394 4218 raft_consensus.cc:729] T 1c71f361364a4e2ea1872308a3e10b7d P 3a1dc6be7d1640ba97604a9c62f597e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a1dc6be7d1640ba97604a9c62f597e9, State: Initialized, Role: FOLLOWER
I20260809 02:02:32.896588 4218 consensus_queue.cc:260] T 1c71f361364a4e2ea1872308a3e10b7d P 3a1dc6be7d1640ba97604a9c62f597e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1dc6be7d1640ba97604a9c62f597e9" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 34563 } } peers { permanent_uuid: "2ea47313d10a4705bf36329be4a87410" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38867 } } peers { permanent_uuid: "ed9f4412136e40cc82447ec2dbda71b9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 43843 } }
I20260809 02:02:32.897346 4223 tablet_replica.cc:439] T 1c71f361364a4e2ea1872308a3e10b7d P 3a1dc6be7d1640ba97604a9c62f597e9: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1dc6be7d1640ba97604a9c62f597e9" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 34563 } } peers { permanent_uuid: "2ea47313d10a4705bf36329be4a87410" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38867 } } peers { permanent_uuid: "ed9f4412136e40cc82447ec2dbda71b9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 43843 } } }
I20260809 02:02:32.897933 4223 tablet_replica.cc:442] T 1c71f361364a4e2ea1872308a3e10b7d P 3a1dc6be7d1640ba97604a9c62f597e9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 02:02:32.898279 4218 ts_tablet_manager.cc:1428] T 1c71f361364a4e2ea1872308a3e10b7d P 3a1dc6be7d1640ba97604a9c62f597e9: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260809 02:02:32.918895 4223 raft_consensus.cc:482] T 1c71f361364a4e2ea1872308a3e10b7d P 3a1dc6be7d1640ba97604a9c62f597e9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:02:32.919032 4223 raft_consensus.cc:504] T 1c71f361364a4e2ea1872308a3e10b7d P 3a1dc6be7d1640ba97604a9c62f597e9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1dc6be7d1640ba97604a9c62f597e9" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 34563 } } peers { permanent_uuid: "2ea47313d10a4705bf36329be4a87410" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38867 } } peers { permanent_uuid: "ed9f4412136e40cc82447ec2dbda71b9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 43843 } }
I20260809 02:02:32.919513 4223 leader_election.cc:288] T 1c71f361364a4e2ea1872308a3e10b7d P 3a1dc6be7d1640ba97604a9c62f597e9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2ea47313d10a4705bf36329be4a87410 (127.1.192.194:38867), ed9f4412136e40cc82447ec2dbda71b9 (127.1.192.195:43843)
I20260809 02:02:32.919962 4136 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c71f361364a4e2ea1872308a3e10b7d" candidate_uuid: "3a1dc6be7d1640ba97604a9c62f597e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed9f4412136e40cc82447ec2dbda71b9" is_pre_election: true
I20260809 02:02:32.919966 3979 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c71f361364a4e2ea1872308a3e10b7d" candidate_uuid: "3a1dc6be7d1640ba97604a9c62f597e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ea47313d10a4705bf36329be4a87410" is_pre_election: true
I20260809 02:02:32.920123 3979 raft_consensus.cc:2454] T 1c71f361364a4e2ea1872308a3e10b7d P 2ea47313d10a4705bf36329be4a87410 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a1dc6be7d1640ba97604a9c62f597e9 in term 0.
I20260809 02:02:32.920128 4136 raft_consensus.cc:2454] T 1c71f361364a4e2ea1872308a3e10b7d P ed9f4412136e40cc82447ec2dbda71b9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a1dc6be7d1640ba97604a9c62f597e9 in term 0.
I20260809 02:02:32.920408 3756 leader_election.cc:302] T 1c71f361364a4e2ea1872308a3e10b7d P 3a1dc6be7d1640ba97604a9c62f597e9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a1dc6be7d1640ba97604a9c62f597e9, ed9f4412136e40cc82447ec2dbda71b9; no voters:
I20260809 02:02:32.920624 4223 raft_consensus.cc:2781] T 1c71f361364a4e2ea1872308a3e10b7d P 3a1dc6be7d1640ba97604a9c62f597e9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:02:32.920691 4223 raft_consensus.cc:482] T 1c71f361364a4e2ea1872308a3e10b7d P 3a1dc6be7d1640ba97604a9c62f597e9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:02:32.920717 4223 raft_consensus.cc:3037] T 1c71f361364a4e2ea1872308a3e10b7d P 3a1dc6be7d1640ba97604a9c62f597e9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:02:32.921779 4223 raft_consensus.cc:504] T 1c71f361364a4e2ea1872308a3e10b7d P 3a1dc6be7d1640ba97604a9c62f597e9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1dc6be7d1640ba97604a9c62f597e9" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 34563 } } peers { permanent_uuid: "2ea47313d10a4705bf36329be4a87410" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38867 } } peers { permanent_uuid: "ed9f4412136e40cc82447ec2dbda71b9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 43843 } }
I20260809 02:02:32.922214 4223 leader_election.cc:288] T 1c71f361364a4e2ea1872308a3e10b7d P 3a1dc6be7d1640ba97604a9c62f597e9 [CANDIDATE]: Term 1 election: Requested vote from peers 2ea47313d10a4705bf36329be4a87410 (127.1.192.194:38867), ed9f4412136e40cc82447ec2dbda71b9 (127.1.192.195:43843)
I20260809 02:02:32.922622 3979 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c71f361364a4e2ea1872308a3e10b7d" candidate_uuid: "3a1dc6be7d1640ba97604a9c62f597e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ea47313d10a4705bf36329be4a87410"
I20260809 02:02:32.922658 4136 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c71f361364a4e2ea1872308a3e10b7d" candidate_uuid: "3a1dc6be7d1640ba97604a9c62f597e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed9f4412136e40cc82447ec2dbda71b9"
I20260809 02:02:32.922758 3979 raft_consensus.cc:3037] T 1c71f361364a4e2ea1872308a3e10b7d P 2ea47313d10a4705bf36329be4a87410 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:02:32.922780 4136 raft_consensus.cc:3037] T 1c71f361364a4e2ea1872308a3e10b7d P ed9f4412136e40cc82447ec2dbda71b9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:02:32.923662 4136 raft_consensus.cc:2454] T 1c71f361364a4e2ea1872308a3e10b7d P ed9f4412136e40cc82447ec2dbda71b9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a1dc6be7d1640ba97604a9c62f597e9 in term 1.
I20260809 02:02:32.923869 3979 raft_consensus.cc:2454] T 1c71f361364a4e2ea1872308a3e10b7d P 2ea47313d10a4705bf36329be4a87410 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a1dc6be7d1640ba97604a9c62f597e9 in term 1.
I20260809 02:02:32.923939 3756 leader_election.cc:302] T 1c71f361364a4e2ea1872308a3e10b7d P 3a1dc6be7d1640ba97604a9c62f597e9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a1dc6be7d1640ba97604a9c62f597e9, ed9f4412136e40cc82447ec2dbda71b9; no voters:
I20260809 02:02:32.924127 4223 raft_consensus.cc:2781] T 1c71f361364a4e2ea1872308a3e10b7d P 3a1dc6be7d1640ba97604a9c62f597e9 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:02:32.924238 4223 raft_consensus.cc:686] T 1c71f361364a4e2ea1872308a3e10b7d P 3a1dc6be7d1640ba97604a9c62f597e9 [term 1 LEADER]: Becoming Leader. State: Replica: 3a1dc6be7d1640ba97604a9c62f597e9, State: Running, Role: LEADER
I20260809 02:02:32.924383 4223 consensus_queue.cc:237] T 1c71f361364a4e2ea1872308a3e10b7d P 3a1dc6be7d1640ba97604a9c62f597e9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1dc6be7d1640ba97604a9c62f597e9" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 34563 } } peers { permanent_uuid: "2ea47313d10a4705bf36329be4a87410" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38867 } } peers { permanent_uuid: "ed9f4412136e40cc82447ec2dbda71b9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 43843 } }
I20260809 02:02:32.925102 4228 tablet_replica.cc:439] T 1c71f361364a4e2ea1872308a3e10b7d P 3a1dc6be7d1640ba97604a9c62f597e9: TxnStatusTablet state changed. Reason: New leader 3a1dc6be7d1640ba97604a9c62f597e9. Latest consensus state: current_term: 1 leader_uuid: "3a1dc6be7d1640ba97604a9c62f597e9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1dc6be7d1640ba97604a9c62f597e9" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 34563 } } peers { permanent_uuid: "2ea47313d10a4705bf36329be4a87410" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38867 } } peers { permanent_uuid: "ed9f4412136e40cc82447ec2dbda71b9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 43843 } } }
I20260809 02:02:32.925210 4228 tablet_replica.cc:442] T 1c71f361364a4e2ea1872308a3e10b7d P 3a1dc6be7d1640ba97604a9c62f597e9: This TxnStatusTablet replica's current role is: LEADER
I20260809 02:02:32.925567 4241 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 02:02:32.925797 4241 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 02:02:32.926186 3659 catalog_manager.cc:5495] T 1c71f361364a4e2ea1872308a3e10b7d P 3a1dc6be7d1640ba97604a9c62f597e9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3a1dc6be7d1640ba97604a9c62f597e9 (127.1.192.193). New cstate: current_term: 1 leader_uuid: "3a1dc6be7d1640ba97604a9c62f597e9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1dc6be7d1640ba97604a9c62f597e9" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 34563 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2ea47313d10a4705bf36329be4a87410" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38867 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ed9f4412136e40cc82447ec2dbda71b9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 43843 } health_report { overall_health: UNKNOWN } } }
W20260809 02:02:32.937614 4192 tablet.cc:2369] T 1c71f361364a4e2ea1872308a3e10b7d P ed9f4412136e40cc82447ec2dbda71b9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 02:02:32.949137 4236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:32.944992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.193:34563 (local address 127.0.0.1:48432)
0809 02:02:32.945254 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:32.945271 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 02:02:32.945389 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:02:32.945813 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:02:32.945817 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:02:32.945824 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 02:02:32.945937 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:32.945950 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:32.946701 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:32.946705 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:32.947397 (+ 692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:32.947407 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:32.947876 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:32.947880 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:32.947977 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:32.948032 (+ 55us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 02:02:32.948734 (+ 702us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 02:02:32.948753 (+ 19us) client_negotiation.cc:769] Sending connection context
0809 02:02:32.948819 (+ 66us) client_negotiation.cc:240] Negotiation successful
0809 02:02:32.948940 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133}
I20260809 02:02:32.949599 4215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:32.945123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48432 (local address 127.1.192.193:34563)
0809 02:02:32.945269 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:32.945274 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:32.945319 (+ 45us) server_negotiation.cc:407] Connection header received
0809 02:02:32.945590 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:32.945593 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:32.945651 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:32.945780 (+ 129us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:02:32.946060 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:32.946576 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:32.947523 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:32.947753 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:32.947864 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:32.948207 (+ 343us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:02:32.948622 (+ 415us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:02:32.948769 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:32.949366 (+ 597us) server_negotiation.cc:299] Negotiation successful
0809 02:02:32.949434 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
W20260809 02:02:32.952530 4029 tablet.cc:2369] T 1c71f361364a4e2ea1872308a3e10b7d P 2ea47313d10a4705bf36329be4a87410: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 02:02:32.954945 4136 raft_consensus.cc:1261] T 1c71f361364a4e2ea1872308a3e10b7d P ed9f4412136e40cc82447ec2dbda71b9 [term 1 FOLLOWER]: Refusing update from remote peer 3a1dc6be7d1640ba97604a9c62f597e9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:02:32.955550 3979 raft_consensus.cc:1261] T 1c71f361364a4e2ea1872308a3e10b7d P 2ea47313d10a4705bf36329be4a87410 [term 1 FOLLOWER]: Refusing update from remote peer 3a1dc6be7d1640ba97604a9c62f597e9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:02:32.955698 4228 consensus_queue.cc:1035] T 1c71f361364a4e2ea1872308a3e10b7d P 3a1dc6be7d1640ba97604a9c62f597e9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed9f4412136e40cc82447ec2dbda71b9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 43843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:02:32.956133 4223 consensus_queue.cc:1035] T 1c71f361364a4e2ea1872308a3e10b7d P 3a1dc6be7d1640ba97604a9c62f597e9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ea47313d10a4705bf36329be4a87410" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38867 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:02:32.959698 4225 tablet_replica.cc:439] T 1c71f361364a4e2ea1872308a3e10b7d P 2ea47313d10a4705bf36329be4a87410: TxnStatusTablet state changed. Reason: New leader 3a1dc6be7d1640ba97604a9c62f597e9. Latest consensus state: current_term: 1 leader_uuid: "3a1dc6be7d1640ba97604a9c62f597e9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1dc6be7d1640ba97604a9c62f597e9" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 34563 } } peers { permanent_uuid: "2ea47313d10a4705bf36329be4a87410" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38867 } } peers { permanent_uuid: "ed9f4412136e40cc82447ec2dbda71b9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 43843 } } }
I20260809 02:02:32.959827 4225 tablet_replica.cc:442] T 1c71f361364a4e2ea1872308a3e10b7d P 2ea47313d10a4705bf36329be4a87410: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 02:02:32.960093 4224 tablet_replica.cc:439] T 1c71f361364a4e2ea1872308a3e10b7d P ed9f4412136e40cc82447ec2dbda71b9: TxnStatusTablet state changed. Reason: New leader 3a1dc6be7d1640ba97604a9c62f597e9. Latest consensus state: current_term: 1 leader_uuid: "3a1dc6be7d1640ba97604a9c62f597e9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1dc6be7d1640ba97604a9c62f597e9" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 34563 } } peers { permanent_uuid: "2ea47313d10a4705bf36329be4a87410" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38867 } } peers { permanent_uuid: "ed9f4412136e40cc82447ec2dbda71b9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 43843 } } }
I20260809 02:02:32.960209 4224 tablet_replica.cc:442] T 1c71f361364a4e2ea1872308a3e10b7d P ed9f4412136e40cc82447ec2dbda71b9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 02:02:32.965127 4245 mvcc.cc:205] Tried to move back new op lower bound from 7316442943295967232 to 7316442943179657216. Current Snapshot: MvccSnapshot[applied={T|T < 7316442943295967232}]
I20260809 02:02:32.965322 4225 tablet_replica.cc:439] T 1c71f361364a4e2ea1872308a3e10b7d P 2ea47313d10a4705bf36329be4a87410: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3a1dc6be7d1640ba97604a9c62f597e9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1dc6be7d1640ba97604a9c62f597e9" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 34563 } } peers { permanent_uuid: "2ea47313d10a4705bf36329be4a87410" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38867 } } peers { permanent_uuid: "ed9f4412136e40cc82447ec2dbda71b9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 43843 } } }
I20260809 02:02:32.965445 4225 tablet_replica.cc:442] T 1c71f361364a4e2ea1872308a3e10b7d P 2ea47313d10a4705bf36329be4a87410: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 02:02:32.965376 4224 tablet_replica.cc:439] T 1c71f361364a4e2ea1872308a3e10b7d P ed9f4412136e40cc82447ec2dbda71b9: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3a1dc6be7d1640ba97604a9c62f597e9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1dc6be7d1640ba97604a9c62f597e9" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 34563 } } peers { permanent_uuid: "2ea47313d10a4705bf36329be4a87410" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38867 } } peers { permanent_uuid: "ed9f4412136e40cc82447ec2dbda71b9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 43843 } } }
I20260809 02:02:32.965523 4224 tablet_replica.cc:442] T 1c71f361364a4e2ea1872308a3e10b7d P ed9f4412136e40cc82447ec2dbda71b9: This TxnStatusTablet replica's current role is: FOLLOWER
W20260809 02:02:32.966434 3881 tablet.cc:2369] T f130ab67eb1d4a8884ab0d0620709b5c P 3a1dc6be7d1640ba97604a9c62f597e9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 02:02:32.966908 4246 mvcc.cc:205] Tried to move back new op lower bound from 7316442943295967232 to 7316442943179657216. Current Snapshot: MvccSnapshot[applied={T|T < 7316442943295967232}]
I20260809 02:02:32.967921 4249 tablet_replica.cc:439] T 1c71f361364a4e2ea1872308a3e10b7d P 3a1dc6be7d1640ba97604a9c62f597e9: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3a1dc6be7d1640ba97604a9c62f597e9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1dc6be7d1640ba97604a9c62f597e9" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 34563 } } peers { permanent_uuid: "2ea47313d10a4705bf36329be4a87410" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38867 } } peers { permanent_uuid: "ed9f4412136e40cc82447ec2dbda71b9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 43843 } } }
I20260809 02:02:32.968073 4249 tablet_replica.cc:442] T 1c71f361364a4e2ea1872308a3e10b7d P 3a1dc6be7d1640ba97604a9c62f597e9: This TxnStatusTablet replica's current role is: LEADER
I20260809 02:02:32.967922 4223 tablet_replica.cc:439] T 1c71f361364a4e2ea1872308a3e10b7d P 3a1dc6be7d1640ba97604a9c62f597e9: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3a1dc6be7d1640ba97604a9c62f597e9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1dc6be7d1640ba97604a9c62f597e9" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 34563 } } peers { permanent_uuid: "2ea47313d10a4705bf36329be4a87410" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38867 } } peers { permanent_uuid: "ed9f4412136e40cc82447ec2dbda71b9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 43843 } } }
I20260809 02:02:32.968274 4223 tablet_replica.cc:442] T 1c71f361364a4e2ea1872308a3e10b7d P 3a1dc6be7d1640ba97604a9c62f597e9: This TxnStatusTablet replica's current role is: LEADER
I20260809 02:02:32.978260 4215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:32.974437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48436 (local address 127.1.192.193:34563)
0809 02:02:32.974554 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:32.974557 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:32.974595 (+ 38us) server_negotiation.cc:407] Connection header received
0809 02:02:32.974742 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:32.974744 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:32.974785 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:32.974848 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:02:32.975132 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:32.975804 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:32.977065 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:32.977321 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:32.977411 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:32.977723 (+ 312us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:02:32.977797 (+ 74us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:02:32.977862 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:32.977985 (+ 123us) server_negotiation.cc:299] Negotiation successful
0809 02:02:32.978037 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 02:02:32.990942 4259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:32.986721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.193:34563 (local address 127.1.192.193:36799)
0809 02:02:32.987095 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:32.987109 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 02:02:32.987218 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:02:32.987462 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:02:32.987465 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:02:32.987472 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 02:02:32.987603 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:32.987609 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:32.988383 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:32.988387 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:32.989393 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:32.989403 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:32.990050 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:32.990054 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:32.990156 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:32.990212 (+ 56us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 02:02:32.990538 (+ 326us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 02:02:32.990544 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 02:02:32.990612 (+ 68us) client_negotiation.cc:240] Negotiation successful
0809 02:02:32.990691 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":102,"threads_started":1}
I20260809 02:02:32.990947 4215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:32.986800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.193:36799 (local address 127.1.192.193:34563)
0809 02:02:32.986915 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:32.986918 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:32.987183 (+ 265us) server_negotiation.cc:407] Connection header received
0809 02:02:32.987322 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:32.987324 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:32.987365 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:32.987421 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:02:32.987743 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:32.988264 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:32.989593 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:32.989917 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:32.990011 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:32.990375 (+ 364us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:02:32.990453 (+ 78us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:02:32.990513 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:32.990664 (+ 151us) server_negotiation.cc:299] Negotiation successful
0809 02:02:32.990741 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 02:02:32.998597 4136 raft_consensus.cc:1261] T f130ab67eb1d4a8884ab0d0620709b5c P ed9f4412136e40cc82447ec2dbda71b9 [term 1 FOLLOWER]: Refusing update from remote peer 3a1dc6be7d1640ba97604a9c62f597e9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:02:32.998761 3979 raft_consensus.cc:1261] T f130ab67eb1d4a8884ab0d0620709b5c P 2ea47313d10a4705bf36329be4a87410 [term 1 FOLLOWER]: Refusing update from remote peer 3a1dc6be7d1640ba97604a9c62f597e9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:02:32.999012 4223 consensus_queue.cc:1035] T f130ab67eb1d4a8884ab0d0620709b5c P 3a1dc6be7d1640ba97604a9c62f597e9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed9f4412136e40cc82447ec2dbda71b9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 43843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:02:32.999274 4228 consensus_queue.cc:1035] T f130ab67eb1d4a8884ab0d0620709b5c P 3a1dc6be7d1640ba97604a9c62f597e9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ea47313d10a4705bf36329be4a87410" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38867 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:02:33.045295 3660 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36318:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:02:33.045535 3660 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36318:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:02:33.048092 3660 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5731d70307644f7ea1a3ccabcb93f4c3: Sending DeleteTablet for 3 replicas of tablet f130ab67eb1d4a8884ab0d0620709b5c
I20260809 02:02:33.048898 4113 tablet_service.cc:1514] Processing DeleteTablet for tablet f130ab67eb1d4a8884ab0d0620709b5c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:02:33 UTC) from {username='kudu', principal='kudu/127.1.192.254@KRBTEST.COM'} at 127.0.0.1:59128
I20260809 02:02:33.049180 3958 tablet_service.cc:1514] Processing DeleteTablet for tablet f130ab67eb1d4a8884ab0d0620709b5c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:02:33 UTC) from {username='kudu', principal='kudu/127.1.192.254@KRBTEST.COM'} at 127.0.0.1:54768
I20260809 02:02:33.049604 4263 tablet_replica.cc:330] T f130ab67eb1d4a8884ab0d0620709b5c P ed9f4412136e40cc82447ec2dbda71b9: stopping tablet replica
I20260809 02:02:33.049639 4264 tablet_replica.cc:330] T f130ab67eb1d4a8884ab0d0620709b5c P 2ea47313d10a4705bf36329be4a87410: stopping tablet replica
I20260809 02:02:33.049683 3810 tablet_service.cc:1514] Processing DeleteTablet for tablet f130ab67eb1d4a8884ab0d0620709b5c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:02:33 UTC) from {username='kudu', principal='kudu/127.1.192.254@KRBTEST.COM'} at 127.0.0.1:48422
I20260809 02:02:33.050066 4264 raft_consensus.cc:2229] T f130ab67eb1d4a8884ab0d0620709b5c P 2ea47313d10a4705bf36329be4a87410 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:02:33.050062 4263 raft_consensus.cc:2229] T f130ab67eb1d4a8884ab0d0620709b5c P ed9f4412136e40cc82447ec2dbda71b9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:02:33.050297 4264 raft_consensus.cc:2258] T f130ab67eb1d4a8884ab0d0620709b5c P 2ea47313d10a4705bf36329be4a87410 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:02:33.050534 4263 raft_consensus.cc:2258] T f130ab67eb1d4a8884ab0d0620709b5c P ed9f4412136e40cc82447ec2dbda71b9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:02:33.050915 4265 tablet_replica.cc:330] T f130ab67eb1d4a8884ab0d0620709b5c P 3a1dc6be7d1640ba97604a9c62f597e9: stopping tablet replica
I20260809 02:02:33.050908 3660 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36318:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 02:02:33.051193 4265 raft_consensus.cc:2229] T f130ab67eb1d4a8884ab0d0620709b5c P 3a1dc6be7d1640ba97604a9c62f597e9 [term 1 LEADER]: Raft consensus shutting down.
W20260809 02:02:33.051340 3660 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:02:33.051564 4265 raft_consensus.cc:2258] T f130ab67eb1d4a8884ab0d0620709b5c P 3a1dc6be7d1640ba97604a9c62f597e9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:02:33.053223 4265 ts_tablet_manager.cc:1905] T f130ab67eb1d4a8884ab0d0620709b5c P 3a1dc6be7d1640ba97604a9c62f597e9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:02:33.053416 4264 ts_tablet_manager.cc:1905] T f130ab67eb1d4a8884ab0d0620709b5c P 2ea47313d10a4705bf36329be4a87410: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:02:33.056689 4265 ts_tablet_manager.cc:1918] T f130ab67eb1d4a8884ab0d0620709b5c P 3a1dc6be7d1640ba97604a9c62f597e9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 02:02:33.056820 4265 log.cc:1196] T f130ab67eb1d4a8884ab0d0620709b5c P 3a1dc6be7d1640ba97604a9c62f597e9: Deleting WAL directory at /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786240941968412-1795-0/minicluster-data/ts-0/wal/wals/f130ab67eb1d4a8884ab0d0620709b5c
I20260809 02:02:33.056924 4113 tablet_service.cc:1467] Processing CreateTablet for tablet 7d320f2e509b48d4991a33933fc64f80 (DEFAULT_TABLE table=test-table [id=ac100d6ca3884ddd9b6d80c6b33a75a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:02:33.057222 4113 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d320f2e509b48d4991a33933fc64f80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:02:33.057253 4265 ts_tablet_manager.cc:1939] T f130ab67eb1d4a8884ab0d0620709b5c P 3a1dc6be7d1640ba97604a9c62f597e9: Deleting consensus metadata
I20260809 02:02:33.057955 4264 ts_tablet_manager.cc:1918] T f130ab67eb1d4a8884ab0d0620709b5c P 2ea47313d10a4705bf36329be4a87410: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 02:02:33.058049 4264 log.cc:1196] T f130ab67eb1d4a8884ab0d0620709b5c P 2ea47313d10a4705bf36329be4a87410: Deleting WAL directory at /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786240941968412-1795-0/minicluster-data/ts-1/wal/wals/f130ab67eb1d4a8884ab0d0620709b5c
I20260809 02:02:33.058368 4264 ts_tablet_manager.cc:1939] T f130ab67eb1d4a8884ab0d0620709b5c P 2ea47313d10a4705bf36329be4a87410: Deleting consensus metadata
I20260809 02:02:33.059724 4216 tablet_bootstrap.cc:492] T 7d320f2e509b48d4991a33933fc64f80 P ed9f4412136e40cc82447ec2dbda71b9: Bootstrap starting.
I20260809 02:02:33.060141 3645 catalog_manager.cc:4841] TS 2ea47313d10a4705bf36329be4a87410 (127.1.192.194:38867): tablet f130ab67eb1d4a8884ab0d0620709b5c (table test-table [id=9c9b7bdade2d4970b57802afd1560fac]) successfully deleted
I20260809 02:02:33.060297 3645 catalog_manager.cc:4841] TS 3a1dc6be7d1640ba97604a9c62f597e9 (127.1.192.193:34563): tablet f130ab67eb1d4a8884ab0d0620709b5c (table test-table [id=9c9b7bdade2d4970b57802afd1560fac]) successfully deleted
I20260809 02:02:33.060714 4216 tablet_bootstrap.cc:654] T 7d320f2e509b48d4991a33933fc64f80 P ed9f4412136e40cc82447ec2dbda71b9: Neither blocks nor log segments found. Creating new log.
I20260809 02:02:33.061578 3810 tablet_service.cc:1467] Processing CreateTablet for tablet 7d320f2e509b48d4991a33933fc64f80 (DEFAULT_TABLE table=test-table [id=ac100d6ca3884ddd9b6d80c6b33a75a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:02:33.061867 3810 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d320f2e509b48d4991a33933fc64f80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:02:33.062000 4216 tablet_bootstrap.cc:492] T 7d320f2e509b48d4991a33933fc64f80 P ed9f4412136e40cc82447ec2dbda71b9: No bootstrap required, opened a new log
I20260809 02:02:33.062088 4216 ts_tablet_manager.cc:1397] T 7d320f2e509b48d4991a33933fc64f80 P ed9f4412136e40cc82447ec2dbda71b9: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 02:02:33.062110 4263 ts_tablet_manager.cc:1905] T f130ab67eb1d4a8884ab0d0620709b5c P ed9f4412136e40cc82447ec2dbda71b9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:02:33.062567 4216 raft_consensus.cc:348] T 7d320f2e509b48d4991a33933fc64f80 P ed9f4412136e40cc82447ec2dbda71b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1dc6be7d1640ba97604a9c62f597e9" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 34563 } } peers { permanent_uuid: "2ea47313d10a4705bf36329be4a87410" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38867 } } peers { permanent_uuid: "ed9f4412136e40cc82447ec2dbda71b9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 43843 } }
I20260809 02:02:33.062721 4216 raft_consensus.cc:374] T 7d320f2e509b48d4991a33933fc64f80 P ed9f4412136e40cc82447ec2dbda71b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:02:33.062764 4216 raft_consensus.cc:729] T 7d320f2e509b48d4991a33933fc64f80 P ed9f4412136e40cc82447ec2dbda71b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed9f4412136e40cc82447ec2dbda71b9, State: Initialized, Role: FOLLOWER
I20260809 02:02:33.062898 4216 consensus_queue.cc:260] T 7d320f2e509b48d4991a33933fc64f80 P ed9f4412136e40cc82447ec2dbda71b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1dc6be7d1640ba97604a9c62f597e9" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 34563 } } peers { permanent_uuid: "2ea47313d10a4705bf36329be4a87410" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38867 } } peers { permanent_uuid: "ed9f4412136e40cc82447ec2dbda71b9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 43843 } }
I20260809 02:02:33.063199 4216 ts_tablet_manager.cc:1428] T 7d320f2e509b48d4991a33933fc64f80 P ed9f4412136e40cc82447ec2dbda71b9: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 02:02:33.064411 4218 tablet_bootstrap.cc:492] T 7d320f2e509b48d4991a33933fc64f80 P 3a1dc6be7d1640ba97604a9c62f597e9: Bootstrap starting.
I20260809 02:02:33.064806 3958 tablet_service.cc:1467] Processing CreateTablet for tablet 7d320f2e509b48d4991a33933fc64f80 (DEFAULT_TABLE table=test-table [id=ac100d6ca3884ddd9b6d80c6b33a75a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:02:33.065104 3958 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d320f2e509b48d4991a33933fc64f80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:02:33.065408 4218 tablet_bootstrap.cc:654] T 7d320f2e509b48d4991a33933fc64f80 P 3a1dc6be7d1640ba97604a9c62f597e9: Neither blocks nor log segments found. Creating new log.
I20260809 02:02:33.067852 4219 tablet_bootstrap.cc:492] T 7d320f2e509b48d4991a33933fc64f80 P 2ea47313d10a4705bf36329be4a87410: Bootstrap starting.
I20260809 02:02:33.068795 4219 tablet_bootstrap.cc:654] T 7d320f2e509b48d4991a33933fc64f80 P 2ea47313d10a4705bf36329be4a87410: Neither blocks nor log segments found. Creating new log.
I20260809 02:02:33.070241 4263 ts_tablet_manager.cc:1918] T f130ab67eb1d4a8884ab0d0620709b5c P ed9f4412136e40cc82447ec2dbda71b9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 02:02:33.070263 4219 tablet_bootstrap.cc:492] T 7d320f2e509b48d4991a33933fc64f80 P 2ea47313d10a4705bf36329be4a87410: No bootstrap required, opened a new log
I20260809 02:02:33.070343 4219 ts_tablet_manager.cc:1397] T 7d320f2e509b48d4991a33933fc64f80 P 2ea47313d10a4705bf36329be4a87410: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 02:02:33.070372 4263 log.cc:1196] T f130ab67eb1d4a8884ab0d0620709b5c P ed9f4412136e40cc82447ec2dbda71b9: Deleting WAL directory at /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786240941968412-1795-0/minicluster-data/ts-2/wal/wals/f130ab67eb1d4a8884ab0d0620709b5c
I20260809 02:02:33.070662 4263 ts_tablet_manager.cc:1939] T f130ab67eb1d4a8884ab0d0620709b5c P ed9f4412136e40cc82447ec2dbda71b9: Deleting consensus metadata
I20260809 02:02:33.070809 4219 raft_consensus.cc:348] T 7d320f2e509b48d4991a33933fc64f80 P 2ea47313d10a4705bf36329be4a87410 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1dc6be7d1640ba97604a9c62f597e9" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 34563 } } peers { permanent_uuid: "2ea47313d10a4705bf36329be4a87410" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38867 } } peers { permanent_uuid: "ed9f4412136e40cc82447ec2dbda71b9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 43843 } }
I20260809 02:02:33.070940 4219 raft_consensus.cc:374] T 7d320f2e509b48d4991a33933fc64f80 P 2ea47313d10a4705bf36329be4a87410 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:02:33.070983 4219 raft_consensus.cc:729] T 7d320f2e509b48d4991a33933fc64f80 P 2ea47313d10a4705bf36329be4a87410 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ea47313d10a4705bf36329be4a87410, State: Initialized, Role: FOLLOWER
I20260809 02:02:33.071131 4219 consensus_queue.cc:260] T 7d320f2e509b48d4991a33933fc64f80 P 2ea47313d10a4705bf36329be4a87410 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1dc6be7d1640ba97604a9c62f597e9" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 34563 } } peers { permanent_uuid: "2ea47313d10a4705bf36329be4a87410" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38867 } } peers { permanent_uuid: "ed9f4412136e40cc82447ec2dbda71b9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 43843 } }
I20260809 02:02:33.071506 4219 ts_tablet_manager.cc:1428] T 7d320f2e509b48d4991a33933fc64f80 P 2ea47313d10a4705bf36329be4a87410: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 02:02:33.072036 3643 catalog_manager.cc:4841] TS ed9f4412136e40cc82447ec2dbda71b9 (127.1.192.195:43843): tablet f130ab67eb1d4a8884ab0d0620709b5c (table test-table [id=9c9b7bdade2d4970b57802afd1560fac]) successfully deleted
I20260809 02:02:33.074390 4218 tablet_bootstrap.cc:492] T 7d320f2e509b48d4991a33933fc64f80 P 3a1dc6be7d1640ba97604a9c62f597e9: No bootstrap required, opened a new log
I20260809 02:02:33.074508 4218 ts_tablet_manager.cc:1397] T 7d320f2e509b48d4991a33933fc64f80 P 3a1dc6be7d1640ba97604a9c62f597e9: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20260809 02:02:33.074916 4218 raft_consensus.cc:348] T 7d320f2e509b48d4991a33933fc64f80 P 3a1dc6be7d1640ba97604a9c62f597e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1dc6be7d1640ba97604a9c62f597e9" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 34563 } } peers { permanent_uuid: "2ea47313d10a4705bf36329be4a87410" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38867 } } peers { permanent_uuid: "ed9f4412136e40cc82447ec2dbda71b9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 43843 } }
I20260809 02:02:33.075048 4218 raft_consensus.cc:374] T 7d320f2e509b48d4991a33933fc64f80 P 3a1dc6be7d1640ba97604a9c62f597e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:02:33.075094 4218 raft_consensus.cc:729] T 7d320f2e509b48d4991a33933fc64f80 P 3a1dc6be7d1640ba97604a9c62f597e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a1dc6be7d1640ba97604a9c62f597e9, State: Initialized, Role: FOLLOWER
I20260809 02:02:33.075232 4218 consensus_queue.cc:260] T 7d320f2e509b48d4991a33933fc64f80 P 3a1dc6be7d1640ba97604a9c62f597e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1dc6be7d1640ba97604a9c62f597e9" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 34563 } } peers { permanent_uuid: "2ea47313d10a4705bf36329be4a87410" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38867 } } peers { permanent_uuid: "ed9f4412136e40cc82447ec2dbda71b9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 43843 } }
I20260809 02:02:33.075517 4218 ts_tablet_manager.cc:1428] T 7d320f2e509b48d4991a33933fc64f80 P 3a1dc6be7d1640ba97604a9c62f597e9: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260809 02:02:33.225090 4225 raft_consensus.cc:482] T 7d320f2e509b48d4991a33933fc64f80 P 2ea47313d10a4705bf36329be4a87410 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:02:33.225261 4225 raft_consensus.cc:504] T 7d320f2e509b48d4991a33933fc64f80 P 2ea47313d10a4705bf36329be4a87410 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1dc6be7d1640ba97604a9c62f597e9" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 34563 } } peers { permanent_uuid: "2ea47313d10a4705bf36329be4a87410" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38867 } } peers { permanent_uuid: "ed9f4412136e40cc82447ec2dbda71b9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 43843 } }
I20260809 02:02:33.226521 4225 leader_election.cc:288] T 7d320f2e509b48d4991a33933fc64f80 P 2ea47313d10a4705bf36329be4a87410 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3a1dc6be7d1640ba97604a9c62f597e9 (127.1.192.193:34563), ed9f4412136e40cc82447ec2dbda71b9 (127.1.192.195:43843)
I20260809 02:02:33.231863 4215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:33.227138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.194:38683 (local address 127.1.192.193:34563)
0809 02:02:33.227256 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:33.227261 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:33.227398 (+ 137us) server_negotiation.cc:407] Connection header received
0809 02:02:33.227598 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:33.227601 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:33.227645 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:33.227693 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:02:33.228096 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:33.228854 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:33.230554 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:33.231167 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:33.231269 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:33.231321 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:33.231652 (+ 331us) server_negotiation.cc:299] Negotiation successful
0809 02:02:33.231734 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 02:02:33.232515 4270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:33.226789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.193:34563 (local address 127.1.192.194:38683)
0809 02:02:33.227301 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:33.227315 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 02:02:33.227476 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:02:33.227824 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:02:33.227828 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:02:33.227846 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:02:33.227966 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:33.227973 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:33.228994 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:33.228997 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:33.230408 (+ 1411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:33.230419 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:33.231415 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:33.231418 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:33.231522 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:33.231543 (+ 21us) client_negotiation.cc:769] Sending connection context
0809 02:02:33.232282 (+ 739us) client_negotiation.cc:240] Negotiation successful
0809 02:02:33.232365 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":73,"threads_started":1}
I20260809 02:02:33.232568 4210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:33.227365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.194:54137 (local address 127.1.192.195:43843)
0809 02:02:33.227477 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:33.227481 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:33.228304 (+ 823us) server_negotiation.cc:407] Connection header received
0809 02:02:33.228470 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:33.228473 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:33.228518 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:33.228564 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:02:33.228890 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:33.229706 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:33.231289 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:33.231887 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:33.231977 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:33.232022 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:33.232366 (+ 344us) server_negotiation.cc:299] Negotiation successful
0809 02:02:33.232443 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 02:02:33.232962 4271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:33.226847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.195:43843 (local address 127.1.192.194:54137)
0809 02:02:33.228230 (+ 1383us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:33.228242 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:02:33.228365 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:02:33.228684 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:02:33.228686 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:02:33.228700 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:02:33.228789 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:33.228794 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:33.229897 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:33.229900 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:33.231125 (+ 1225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:33.231135 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:33.232112 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:33.232115 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:33.232236 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:33.232254 (+ 18us) client_negotiation.cc:769] Sending connection context
0809 02:02:33.232753 (+ 499us) client_negotiation.cc:240] Negotiation successful
0809 02:02:33.232830 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1323,"thread_start_us":118,"threads_started":1}
I20260809 02:02:33.233287 3831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d320f2e509b48d4991a33933fc64f80" candidate_uuid: "2ea47313d10a4705bf36329be4a87410" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a1dc6be7d1640ba97604a9c62f597e9" is_pre_election: true
I20260809 02:02:33.233371 4135 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d320f2e509b48d4991a33933fc64f80" candidate_uuid: "2ea47313d10a4705bf36329be4a87410" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed9f4412136e40cc82447ec2dbda71b9" is_pre_election: true
I20260809 02:02:33.233527 4135 raft_consensus.cc:2454] T 7d320f2e509b48d4991a33933fc64f80 P ed9f4412136e40cc82447ec2dbda71b9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ea47313d10a4705bf36329be4a87410 in term 0.
I20260809 02:02:33.233541 3831 raft_consensus.cc:2454] T 7d320f2e509b48d4991a33933fc64f80 P 3a1dc6be7d1640ba97604a9c62f597e9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ea47313d10a4705bf36329be4a87410 in term 0.
I20260809 02:02:33.233906 3906 leader_election.cc:302] T 7d320f2e509b48d4991a33933fc64f80 P 2ea47313d10a4705bf36329be4a87410 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ea47313d10a4705bf36329be4a87410, ed9f4412136e40cc82447ec2dbda71b9; no voters:
I20260809 02:02:33.234182 4225 raft_consensus.cc:2781] T 7d320f2e509b48d4991a33933fc64f80 P 2ea47313d10a4705bf36329be4a87410 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:02:33.234234 4225 raft_consensus.cc:482] T 7d320f2e509b48d4991a33933fc64f80 P 2ea47313d10a4705bf36329be4a87410 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:02:33.234269 4225 raft_consensus.cc:3037] T 7d320f2e509b48d4991a33933fc64f80 P 2ea47313d10a4705bf36329be4a87410 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:02:33.235661 4225 raft_consensus.cc:504] T 7d320f2e509b48d4991a33933fc64f80 P 2ea47313d10a4705bf36329be4a87410 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1dc6be7d1640ba97604a9c62f597e9" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 34563 } } peers { permanent_uuid: "2ea47313d10a4705bf36329be4a87410" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38867 } } peers { permanent_uuid: "ed9f4412136e40cc82447ec2dbda71b9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 43843 } }
I20260809 02:02:33.236099 4225 leader_election.cc:288] T 7d320f2e509b48d4991a33933fc64f80 P 2ea47313d10a4705bf36329be4a87410 [CANDIDATE]: Term 1 election: Requested vote from peers 3a1dc6be7d1640ba97604a9c62f597e9 (127.1.192.193:34563), ed9f4412136e40cc82447ec2dbda71b9 (127.1.192.195:43843)
I20260809 02:02:33.236467 3831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d320f2e509b48d4991a33933fc64f80" candidate_uuid: "2ea47313d10a4705bf36329be4a87410" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a1dc6be7d1640ba97604a9c62f597e9"
I20260809 02:02:33.236470 4135 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d320f2e509b48d4991a33933fc64f80" candidate_uuid: "2ea47313d10a4705bf36329be4a87410" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed9f4412136e40cc82447ec2dbda71b9"
I20260809 02:02:33.236595 4135 raft_consensus.cc:3037] T 7d320f2e509b48d4991a33933fc64f80 P ed9f4412136e40cc82447ec2dbda71b9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:02:33.236595 3831 raft_consensus.cc:3037] T 7d320f2e509b48d4991a33933fc64f80 P 3a1dc6be7d1640ba97604a9c62f597e9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:02:33.237825 4135 raft_consensus.cc:2454] T 7d320f2e509b48d4991a33933fc64f80 P ed9f4412136e40cc82447ec2dbda71b9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ea47313d10a4705bf36329be4a87410 in term 1.
I20260809 02:02:33.237825 3831 raft_consensus.cc:2454] T 7d320f2e509b48d4991a33933fc64f80 P 3a1dc6be7d1640ba97604a9c62f597e9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ea47313d10a4705bf36329be4a87410 in term 1.
I20260809 02:02:33.238108 3906 leader_election.cc:302] T 7d320f2e509b48d4991a33933fc64f80 P 2ea47313d10a4705bf36329be4a87410 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ea47313d10a4705bf36329be4a87410, ed9f4412136e40cc82447ec2dbda71b9; no voters:
I20260809 02:02:33.238258 4225 raft_consensus.cc:2781] T 7d320f2e509b48d4991a33933fc64f80 P 2ea47313d10a4705bf36329be4a87410 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:02:33.238524 4225 raft_consensus.cc:686] T 7d320f2e509b48d4991a33933fc64f80 P 2ea47313d10a4705bf36329be4a87410 [term 1 LEADER]: Becoming Leader. State: Replica: 2ea47313d10a4705bf36329be4a87410, State: Running, Role: LEADER
I20260809 02:02:33.238821 4225 consensus_queue.cc:237] T 7d320f2e509b48d4991a33933fc64f80 P 2ea47313d10a4705bf36329be4a87410 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1dc6be7d1640ba97604a9c62f597e9" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 34563 } } peers { permanent_uuid: "2ea47313d10a4705bf36329be4a87410" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38867 } } peers { permanent_uuid: "ed9f4412136e40cc82447ec2dbda71b9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 43843 } }
I20260809 02:02:33.241793 3659 catalog_manager.cc:5495] T 7d320f2e509b48d4991a33933fc64f80 P 2ea47313d10a4705bf36329be4a87410 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2ea47313d10a4705bf36329be4a87410 (127.1.192.194). New cstate: current_term: 1 leader_uuid: "2ea47313d10a4705bf36329be4a87410" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1dc6be7d1640ba97604a9c62f597e9" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 34563 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ea47313d10a4705bf36329be4a87410" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38867 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ed9f4412136e40cc82447ec2dbda71b9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 43843 } health_report { overall_health: UNKNOWN } } }
I20260809 02:02:33.267762 4211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:33.264008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54774 (local address 127.1.192.194:38867)
0809 02:02:33.264130 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:33.264134 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:33.264148 (+ 14us) server_negotiation.cc:407] Connection header received
0809 02:02:33.264330 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:33.264334 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:33.264394 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:33.264510 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:02:33.264751 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:33.265265 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:33.266187 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:33.266424 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:33.266540 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:33.266826 (+ 286us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:02:33.267304 (+ 478us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:02:33.267432 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:33.267547 (+ 115us) server_negotiation.cc:299] Negotiation successful
0809 02:02:33.267615 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 02:02:33.279455 4215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:33.275692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.194:46699 (local address 127.1.192.193:34563)
0809 02:02:33.275830 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:33.275834 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:33.275916 (+ 82us) server_negotiation.cc:407] Connection header received
0809 02:02:33.276070 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:33.276072 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:33.276123 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:33.276241 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:02:33.276564 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:33.277138 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:33.278212 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:33.278467 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:33.278599 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:33.278952 (+ 353us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:02:33.279031 (+ 79us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:02:33.279119 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:33.279225 (+ 106us) server_negotiation.cc:299] Negotiation successful
0809 02:02:33.279288 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 02:02:33.279785 4276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:33.275505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.193:34563 (local address 127.1.192.194:46699)
0809 02:02:33.275825 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:33.275839 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 02:02:33.275939 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:02:33.276254 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:02:33.276258 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:02:33.276267 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 02:02:33.276426 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:33.276433 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:33.277271 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:33.277275 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:33.278071 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:33.278081 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:33.278616 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:33.278620 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:33.278732 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:33.278781 (+ 49us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 02:02:33.279124 (+ 343us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 02:02:33.279134 (+ 10us) client_negotiation.cc:769] Sending connection context
0809 02:02:33.279203 (+ 69us) client_negotiation.cc:240] Negotiation successful
0809 02:02:33.279317 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":111,"threads_started":1}
I20260809 02:02:33.286317 4135 raft_consensus.cc:1261] T 7d320f2e509b48d4991a33933fc64f80 P ed9f4412136e40cc82447ec2dbda71b9 [term 1 FOLLOWER]: Refusing update from remote peer 2ea47313d10a4705bf36329be4a87410: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:02:33.286563 3831 raft_consensus.cc:1261] T 7d320f2e509b48d4991a33933fc64f80 P 3a1dc6be7d1640ba97604a9c62f597e9 [term 1 FOLLOWER]: Refusing update from remote peer 2ea47313d10a4705bf36329be4a87410: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:02:33.286854 4272 consensus_queue.cc:1035] T 7d320f2e509b48d4991a33933fc64f80 P 2ea47313d10a4705bf36329be4a87410 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed9f4412136e40cc82447ec2dbda71b9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 43843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:02:33.287133 4225 consensus_queue.cc:1035] T 7d320f2e509b48d4991a33933fc64f80 P 2ea47313d10a4705bf36329be4a87410 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a1dc6be7d1640ba97604a9c62f597e9" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 34563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:02:33.291438 4244 mvcc.cc:205] Tried to move back new op lower bound from 7316442944656412672 to 7316442944468566016. Current Snapshot: MvccSnapshot[applied={T|T < 7316442944656412672}]
I20260809 02:02:33.305533 4211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:33.302234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.193:60051 (local address 127.1.192.194:38867)
0809 02:02:33.302374 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:33.302379 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:33.302395 (+ 16us) server_negotiation.cc:407] Connection header received
0809 02:02:33.302442 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:33.302445 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:33.302495 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:33.302589 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:02:33.302823 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:33.303566 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:33.304444 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:33.304687 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:33.304796 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:33.305055 (+ 259us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:02:33.305139 (+ 84us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:02:33.305220 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:33.305313 (+ 93us) server_negotiation.cc:299] Negotiation successful
0809 02:02:33.305364 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 02:02:33.305527 4259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:33.302006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.194:38867 (local address 127.1.192.193:60051)
0809 02:02:33.302142 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:33.302184 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 02:02:33.302277 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:02:33.302603 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:02:33.302606 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:02:33.302614 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 02:02:33.302728 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:33.302736 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:33.303713 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:33.303718 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:33.304318 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:33.304326 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:33.304806 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:33.304809 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:33.304896 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:33.304932 (+ 36us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 02:02:33.305225 (+ 293us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 02:02:33.305234 (+ 9us) client_negotiation.cc:769] Sending connection context
0809 02:02:33.305302 (+ 68us) client_negotiation.cc:240] Negotiation successful
0809 02:02:33.305390 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
W20260809 02:02:33.306553 4246 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260809 02:02:34.881345 3659 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36318:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:02:34.881568 3659 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36318:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:02:34.884485 3659 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5731d70307644f7ea1a3ccabcb93f4c3: Sending DeleteTablet for 3 replicas of tablet 7d320f2e509b48d4991a33933fc64f80
I20260809 02:02:34.885172 3958 tablet_service.cc:1514] Processing DeleteTablet for tablet 7d320f2e509b48d4991a33933fc64f80 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:02:34 UTC) from {username='kudu', principal='kudu/127.1.192.254@KRBTEST.COM'} at 127.0.0.1:54768
I20260809 02:02:34.885201 3810 tablet_service.cc:1514] Processing DeleteTablet for tablet 7d320f2e509b48d4991a33933fc64f80 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:02:34 UTC) from {username='kudu', principal='kudu/127.1.192.254@KRBTEST.COM'} at 127.0.0.1:48422
I20260809 02:02:34.885352 4113 tablet_service.cc:1514] Processing DeleteTablet for tablet 7d320f2e509b48d4991a33933fc64f80 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:02:34 UTC) from {username='kudu', principal='kudu/127.1.192.254@KRBTEST.COM'} at 127.0.0.1:59128
I20260809 02:02:34.885649 4305 tablet_replica.cc:330] T 7d320f2e509b48d4991a33933fc64f80 P 3a1dc6be7d1640ba97604a9c62f597e9: stopping tablet replica
I20260809 02:02:34.885650 4304 tablet_replica.cc:330] T 7d320f2e509b48d4991a33933fc64f80 P 2ea47313d10a4705bf36329be4a87410: stopping tablet replica
I20260809 02:02:34.885743 4306 tablet_replica.cc:330] T 7d320f2e509b48d4991a33933fc64f80 P ed9f4412136e40cc82447ec2dbda71b9: stopping tablet replica
I20260809 02:02:34.885802 4304 raft_consensus.cc:2229] T 7d320f2e509b48d4991a33933fc64f80 P 2ea47313d10a4705bf36329be4a87410 [term 1 LEADER]: Raft consensus shutting down.
I20260809 02:02:34.885874 4306 raft_consensus.cc:2229] T 7d320f2e509b48d4991a33933fc64f80 P ed9f4412136e40cc82447ec2dbda71b9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:02:34.886011 4306 raft_consensus.cc:2258] T 7d320f2e509b48d4991a33933fc64f80 P ed9f4412136e40cc82447ec2dbda71b9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:02:34.886196 4304 raft_consensus.cc:2258] T 7d320f2e509b48d4991a33933fc64f80 P 2ea47313d10a4705bf36329be4a87410 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:02:34.886188 4305 raft_consensus.cc:2229] T 7d320f2e509b48d4991a33933fc64f80 P 3a1dc6be7d1640ba97604a9c62f597e9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:02:34.886395 4305 raft_consensus.cc:2258] T 7d320f2e509b48d4991a33933fc64f80 P 3a1dc6be7d1640ba97604a9c62f597e9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:02:34.887068 4306 ts_tablet_manager.cc:1905] T 7d320f2e509b48d4991a33933fc64f80 P ed9f4412136e40cc82447ec2dbda71b9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:02:34.887318 4305 ts_tablet_manager.cc:1905] T 7d320f2e509b48d4991a33933fc64f80 P 3a1dc6be7d1640ba97604a9c62f597e9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:02:34.890477 4306 ts_tablet_manager.cc:1918] T 7d320f2e509b48d4991a33933fc64f80 P ed9f4412136e40cc82447ec2dbda71b9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 02:02:34.890568 4306 log.cc:1196] T 7d320f2e509b48d4991a33933fc64f80 P ed9f4412136e40cc82447ec2dbda71b9: Deleting WAL directory at /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786240941968412-1795-0/minicluster-data/ts-2/wal/wals/7d320f2e509b48d4991a33933fc64f80
I20260809 02:02:34.890583 4304 ts_tablet_manager.cc:1905] T 7d320f2e509b48d4991a33933fc64f80 P 2ea47313d10a4705bf36329be4a87410: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:02:34.890877 4306 ts_tablet_manager.cc:1939] T 7d320f2e509b48d4991a33933fc64f80 P ed9f4412136e40cc82447ec2dbda71b9: Deleting consensus metadata
I20260809 02:02:34.891044 4305 ts_tablet_manager.cc:1918] T 7d320f2e509b48d4991a33933fc64f80 P 3a1dc6be7d1640ba97604a9c62f597e9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 02:02:34.891186 4305 log.cc:1196] T 7d320f2e509b48d4991a33933fc64f80 P 3a1dc6be7d1640ba97604a9c62f597e9: Deleting WAL directory at /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786240941968412-1795-0/minicluster-data/ts-0/wal/wals/7d320f2e509b48d4991a33933fc64f80
I20260809 02:02:34.891556 4305 ts_tablet_manager.cc:1939] T 7d320f2e509b48d4991a33933fc64f80 P 3a1dc6be7d1640ba97604a9c62f597e9: Deleting consensus metadata
I20260809 02:02:34.891714 3643 catalog_manager.cc:4841] TS ed9f4412136e40cc82447ec2dbda71b9 (127.1.192.195:43843): tablet 7d320f2e509b48d4991a33933fc64f80 (table test-table [id=ac100d6ca3884ddd9b6d80c6b33a75a8]) successfully deleted
I20260809 02:02:34.892499 3645 catalog_manager.cc:4841] TS 3a1dc6be7d1640ba97604a9c62f597e9 (127.1.192.193:34563): tablet 7d320f2e509b48d4991a33933fc64f80 (table test-table [id=ac100d6ca3884ddd9b6d80c6b33a75a8]) successfully deleted
I20260809 02:02:34.894019 4304 ts_tablet_manager.cc:1918] T 7d320f2e509b48d4991a33933fc64f80 P 2ea47313d10a4705bf36329be4a87410: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 02:02:34.894124 4304 log.cc:1196] T 7d320f2e509b48d4991a33933fc64f80 P 2ea47313d10a4705bf36329be4a87410: Deleting WAL directory at /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786240941968412-1795-0/minicluster-data/ts-1/wal/wals/7d320f2e509b48d4991a33933fc64f80
I20260809 02:02:34.894431 4304 ts_tablet_manager.cc:1939] T 7d320f2e509b48d4991a33933fc64f80 P 2ea47313d10a4705bf36329be4a87410: Deleting consensus metadata
I20260809 02:02:34.895126 3645 catalog_manager.cc:4841] TS 2ea47313d10a4705bf36329be4a87410 (127.1.192.194:38867): tablet 7d320f2e509b48d4991a33933fc64f80 (table test-table [id=ac100d6ca3884ddd9b6d80c6b33a75a8]) successfully deleted
Aug 09 02:02:34 dist-test-slave-ss4n krb5kdc[3613](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240954, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 02:02:34 dist-test-slave-ss4n krb5kdc[3613](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240954, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.192.254@KRBTEST.COM
I20260809 02:02:34.919762 4315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:34.906400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36344 (local address 127.1.192.254:43253)
0809 02:02:34.906774 (+ 374us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:34.906781 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:34.906797 (+ 16us) server_negotiation.cc:407] Connection header received
0809 02:02:34.906935 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:34.906938 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:34.906981 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:34.907061 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:34.907775 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:34.908337 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:34.909132 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:34.909345 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:34.909421 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:34.915989 (+ 6568us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:02:34.915993 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:02:34.915994 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:02:34.916025 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:02:34.917823 (+ 1798us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:34.918324 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:34.918328 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:34.918330 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:34.918425 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:34.918800 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:34.918805 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:34.918807 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:34.919018 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:02:34.919168 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:34.919429 (+ 261us) server_negotiation.cc:299] Negotiation successful
0809 02:02:34.919567 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":120,"threads_started":1}
I20260809 02:02:34.922729 3659 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36344:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 02:02:34.923224 3659 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:02:34.928578 3958 tablet_service.cc:1467] Processing CreateTablet for tablet a792ee4c209b4db9a22f42d65729a2ac (DEFAULT_TABLE table=test-table [id=507b9ffef78741c1854c6981641cca46]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:02:34.928599 3810 tablet_service.cc:1467] Processing CreateTablet for tablet a792ee4c209b4db9a22f42d65729a2ac (DEFAULT_TABLE table=test-table [id=507b9ffef78741c1854c6981641cca46]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:02:34.928731 4113 tablet_service.cc:1467] Processing CreateTablet for tablet a792ee4c209b4db9a22f42d65729a2ac (DEFAULT_TABLE table=test-table [id=507b9ffef78741c1854c6981641cca46]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:02:34.928890 3958 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a792ee4c209b4db9a22f42d65729a2ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:02:34.928928 3810 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a792ee4c209b4db9a22f42d65729a2ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:02:34.929015 4113 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a792ee4c209b4db9a22f42d65729a2ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:02:34.931346 4316 tablet_bootstrap.cc:492] T a792ee4c209b4db9a22f42d65729a2ac P 2ea47313d10a4705bf36329be4a87410: Bootstrap starting.
I20260809 02:02:34.931777 4317 tablet_bootstrap.cc:492] T a792ee4c209b4db9a22f42d65729a2ac P ed9f4412136e40cc82447ec2dbda71b9: Bootstrap starting.
I20260809 02:02:34.931777 4318 tablet_bootstrap.cc:492] T a792ee4c209b4db9a22f42d65729a2ac P 3a1dc6be7d1640ba97604a9c62f597e9: Bootstrap starting.
I20260809 02:02:34.932744 4317 tablet_bootstrap.cc:654] T a792ee4c209b4db9a22f42d65729a2ac P ed9f4412136e40cc82447ec2dbda71b9: Neither blocks nor log segments found. Creating new log.
I20260809 02:02:34.932744 4318 tablet_bootstrap.cc:654] T a792ee4c209b4db9a22f42d65729a2ac P 3a1dc6be7d1640ba97604a9c62f597e9: Neither blocks nor log segments found. Creating new log.
I20260809 02:02:34.933063 4316 tablet_bootstrap.cc:654] T a792ee4c209b4db9a22f42d65729a2ac P 2ea47313d10a4705bf36329be4a87410: Neither blocks nor log segments found. Creating new log.
I20260809 02:02:34.934074 4318 tablet_bootstrap.cc:492] T a792ee4c209b4db9a22f42d65729a2ac P 3a1dc6be7d1640ba97604a9c62f597e9: No bootstrap required, opened a new log
I20260809 02:02:34.934091 4317 tablet_bootstrap.cc:492] T a792ee4c209b4db9a22f42d65729a2ac P ed9f4412136e40cc82447ec2dbda71b9: No bootstrap required, opened a new log
I20260809 02:02:34.934188 4318 ts_tablet_manager.cc:1397] T a792ee4c209b4db9a22f42d65729a2ac P 3a1dc6be7d1640ba97604a9c62f597e9: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 02:02:34.934206 4317 ts_tablet_manager.cc:1397] T a792ee4c209b4db9a22f42d65729a2ac P ed9f4412136e40cc82447ec2dbda71b9: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 02:02:34.934353 4316 tablet_bootstrap.cc:492] T a792ee4c209b4db9a22f42d65729a2ac P 2ea47313d10a4705bf36329be4a87410: No bootstrap required, opened a new log
I20260809 02:02:34.934448 4316 ts_tablet_manager.cc:1397] T a792ee4c209b4db9a22f42d65729a2ac P 2ea47313d10a4705bf36329be4a87410: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 02:02:34.934643 4318 raft_consensus.cc:348] T a792ee4c209b4db9a22f42d65729a2ac P 3a1dc6be7d1640ba97604a9c62f597e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1dc6be7d1640ba97604a9c62f597e9" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 34563 } } peers { permanent_uuid: "2ea47313d10a4705bf36329be4a87410" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38867 } } peers { permanent_uuid: "ed9f4412136e40cc82447ec2dbda71b9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 43843 } }
I20260809 02:02:34.934764 4318 raft_consensus.cc:374] T a792ee4c209b4db9a22f42d65729a2ac P 3a1dc6be7d1640ba97604a9c62f597e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:02:34.934690 4317 raft_consensus.cc:348] T a792ee4c209b4db9a22f42d65729a2ac P ed9f4412136e40cc82447ec2dbda71b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1dc6be7d1640ba97604a9c62f597e9" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 34563 } } peers { permanent_uuid: "2ea47313d10a4705bf36329be4a87410" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38867 } } peers { permanent_uuid: "ed9f4412136e40cc82447ec2dbda71b9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 43843 } }
I20260809 02:02:34.934800 4318 raft_consensus.cc:729] T a792ee4c209b4db9a22f42d65729a2ac P 3a1dc6be7d1640ba97604a9c62f597e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a1dc6be7d1640ba97604a9c62f597e9, State: Initialized, Role: FOLLOWER
I20260809 02:02:34.934810 4317 raft_consensus.cc:374] T a792ee4c209b4db9a22f42d65729a2ac P ed9f4412136e40cc82447ec2dbda71b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:02:34.934849 4317 raft_consensus.cc:729] T a792ee4c209b4db9a22f42d65729a2ac P ed9f4412136e40cc82447ec2dbda71b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed9f4412136e40cc82447ec2dbda71b9, State: Initialized, Role: FOLLOWER
I20260809 02:02:34.934860 4316 raft_consensus.cc:348] T a792ee4c209b4db9a22f42d65729a2ac P 2ea47313d10a4705bf36329be4a87410 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1dc6be7d1640ba97604a9c62f597e9" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 34563 } } peers { permanent_uuid: "2ea47313d10a4705bf36329be4a87410" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38867 } } peers { permanent_uuid: "ed9f4412136e40cc82447ec2dbda71b9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 43843 } }
I20260809 02:02:34.934963 4316 raft_consensus.cc:374] T a792ee4c209b4db9a22f42d65729a2ac P 2ea47313d10a4705bf36329be4a87410 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:02:34.934929 4318 consensus_queue.cc:260] T a792ee4c209b4db9a22f42d65729a2ac P 3a1dc6be7d1640ba97604a9c62f597e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1dc6be7d1640ba97604a9c62f597e9" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 34563 } } peers { permanent_uuid: "2ea47313d10a4705bf36329be4a87410" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38867 } } peers { permanent_uuid: "ed9f4412136e40cc82447ec2dbda71b9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 43843 } }
I20260809 02:02:34.934995 4316 raft_consensus.cc:729] T a792ee4c209b4db9a22f42d65729a2ac P 2ea47313d10a4705bf36329be4a87410 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ea47313d10a4705bf36329be4a87410, State: Initialized, Role: FOLLOWER
I20260809 02:02:34.934996 4317 consensus_queue.cc:260] T a792ee4c209b4db9a22f42d65729a2ac P ed9f4412136e40cc82447ec2dbda71b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1dc6be7d1640ba97604a9c62f597e9" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 34563 } } peers { permanent_uuid: "2ea47313d10a4705bf36329be4a87410" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38867 } } peers { permanent_uuid: "ed9f4412136e40cc82447ec2dbda71b9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 43843 } }
I20260809 02:02:34.935113 4316 consensus_queue.cc:260] T a792ee4c209b4db9a22f42d65729a2ac P 2ea47313d10a4705bf36329be4a87410 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1dc6be7d1640ba97604a9c62f597e9" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 34563 } } peers { permanent_uuid: "2ea47313d10a4705bf36329be4a87410" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38867 } } peers { permanent_uuid: "ed9f4412136e40cc82447ec2dbda71b9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 43843 } }
I20260809 02:02:34.935236 4318 ts_tablet_manager.cc:1428] T a792ee4c209b4db9a22f42d65729a2ac P 3a1dc6be7d1640ba97604a9c62f597e9: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 02:02:34.935390 4316 ts_tablet_manager.cc:1428] T a792ee4c209b4db9a22f42d65729a2ac P 2ea47313d10a4705bf36329be4a87410: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 02:02:34.935923 4317 ts_tablet_manager.cc:1428] T a792ee4c209b4db9a22f42d65729a2ac P ed9f4412136e40cc82447ec2dbda71b9: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 02:02:34.954499 4322 raft_consensus.cc:482] T a792ee4c209b4db9a22f42d65729a2ac P ed9f4412136e40cc82447ec2dbda71b9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:02:34.954650 4322 raft_consensus.cc:504] T a792ee4c209b4db9a22f42d65729a2ac P ed9f4412136e40cc82447ec2dbda71b9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1dc6be7d1640ba97604a9c62f597e9" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 34563 } } peers { permanent_uuid: "2ea47313d10a4705bf36329be4a87410" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38867 } } peers { permanent_uuid: "ed9f4412136e40cc82447ec2dbda71b9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 43843 } }
I20260809 02:02:34.955886 4322 leader_election.cc:288] T a792ee4c209b4db9a22f42d65729a2ac P ed9f4412136e40cc82447ec2dbda71b9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3a1dc6be7d1640ba97604a9c62f597e9 (127.1.192.193:34563), 2ea47313d10a4705bf36329be4a87410 (127.1.192.194:38867)
I20260809 02:02:34.960311 4323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:34.956033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.193:34563 (local address 127.1.192.195:53673)
0809 02:02:34.956433 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:34.956449 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 02:02:34.956576 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:02:34.956997 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:02:34.957000 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:02:34.957040 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:02:34.957189 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:34.957199 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:34.958077 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:34.958081 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:34.958982 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:34.958991 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:34.959828 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:34.959832 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:34.959947 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:34.959973 (+ 26us) client_negotiation.cc:769] Sending connection context
0809 02:02:34.960044 (+ 71us) client_negotiation.cc:240] Negotiation successful
0809 02:02:34.960125 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":311,"thread_start_us":159,"threads_started":1}
I20260809 02:02:34.960557 4326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:34.956615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.195:59733 (local address 127.1.192.194:38867)
0809 02:02:34.956859 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:34.956866 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:34.956925 (+ 59us) server_negotiation.cc:407] Connection header received
0809 02:02:34.957099 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:34.957102 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:34.957154 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:34.957251 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:02:34.957552 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:34.958129 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:34.959236 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:34.959826 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:34.959945 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:34.960016 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:34.960201 (+ 185us) server_negotiation.cc:299] Negotiation successful
0809 02:02:34.960278 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":93,"threads_started":1}
I20260809 02:02:34.960597 4324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:34.956154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.195:53673 (local address 127.1.192.193:34563)
0809 02:02:34.956464 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:34.956475 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:34.956518 (+ 43us) server_negotiation.cc:407] Connection header received
0809 02:02:34.956775 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:34.956779 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:34.956834 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:34.956980 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:02:34.957297 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:34.957963 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:34.959131 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:34.959692 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:34.959837 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:34.959895 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:34.960392 (+ 497us) server_negotiation.cc:299] Negotiation successful
0809 02:02:34.960486 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":229,"thread_start_us":125,"threads_started":1}
I20260809 02:02:34.960325 4325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:34.956515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.194:38867 (local address 127.1.192.195:59733)
0809 02:02:34.956825 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:34.956837 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:02:34.956976 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:02:34.957273 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:02:34.957278 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:02:34.957296 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:02:34.957415 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:34.957422 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:34.958275 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:34.958277 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:34.959074 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:34.959089 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:34.959973 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:34.959976 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:34.960081 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:34.960100 (+ 19us) client_negotiation.cc:769] Sending connection context
0809 02:02:34.960167 (+ 67us) client_negotiation.cc:240] Negotiation successful
0809 02:02:34.960225 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":89,"threads_started":1}
I20260809 02:02:34.961179 3831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a792ee4c209b4db9a22f42d65729a2ac" candidate_uuid: "ed9f4412136e40cc82447ec2dbda71b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a1dc6be7d1640ba97604a9c62f597e9" is_pre_election: true
I20260809 02:02:34.961231 3979 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a792ee4c209b4db9a22f42d65729a2ac" candidate_uuid: "ed9f4412136e40cc82447ec2dbda71b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ea47313d10a4705bf36329be4a87410" is_pre_election: true
I20260809 02:02:34.961351 3831 raft_consensus.cc:2454] T a792ee4c209b4db9a22f42d65729a2ac P 3a1dc6be7d1640ba97604a9c62f597e9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ed9f4412136e40cc82447ec2dbda71b9 in term 0.
I20260809 02:02:34.961386 3979 raft_consensus.cc:2454] T a792ee4c209b4db9a22f42d65729a2ac P 2ea47313d10a4705bf36329be4a87410 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ed9f4412136e40cc82447ec2dbda71b9 in term 0.
I20260809 02:02:34.961777 4059 leader_election.cc:302] T a792ee4c209b4db9a22f42d65729a2ac P ed9f4412136e40cc82447ec2dbda71b9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ea47313d10a4705bf36329be4a87410, ed9f4412136e40cc82447ec2dbda71b9; no voters:
I20260809 02:02:34.962065 4322 raft_consensus.cc:2781] T a792ee4c209b4db9a22f42d65729a2ac P ed9f4412136e40cc82447ec2dbda71b9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:02:34.962132 4322 raft_consensus.cc:482] T a792ee4c209b4db9a22f42d65729a2ac P ed9f4412136e40cc82447ec2dbda71b9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:02:34.962184 4322 raft_consensus.cc:3037] T a792ee4c209b4db9a22f42d65729a2ac P ed9f4412136e40cc82447ec2dbda71b9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:02:34.963236 4322 raft_consensus.cc:504] T a792ee4c209b4db9a22f42d65729a2ac P ed9f4412136e40cc82447ec2dbda71b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1dc6be7d1640ba97604a9c62f597e9" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 34563 } } peers { permanent_uuid: "2ea47313d10a4705bf36329be4a87410" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38867 } } peers { permanent_uuid: "ed9f4412136e40cc82447ec2dbda71b9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 43843 } }
I20260809 02:02:34.963595 4322 leader_election.cc:288] T a792ee4c209b4db9a22f42d65729a2ac P ed9f4412136e40cc82447ec2dbda71b9 [CANDIDATE]: Term 1 election: Requested vote from peers 3a1dc6be7d1640ba97604a9c62f597e9 (127.1.192.193:34563), 2ea47313d10a4705bf36329be4a87410 (127.1.192.194:38867)
I20260809 02:02:34.963986 3831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a792ee4c209b4db9a22f42d65729a2ac" candidate_uuid: "ed9f4412136e40cc82447ec2dbda71b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a1dc6be7d1640ba97604a9c62f597e9"
I20260809 02:02:34.963986 3979 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a792ee4c209b4db9a22f42d65729a2ac" candidate_uuid: "ed9f4412136e40cc82447ec2dbda71b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ea47313d10a4705bf36329be4a87410"
I20260809 02:02:34.964120 3831 raft_consensus.cc:3037] T a792ee4c209b4db9a22f42d65729a2ac P 3a1dc6be7d1640ba97604a9c62f597e9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:02:34.964115 3979 raft_consensus.cc:3037] T a792ee4c209b4db9a22f42d65729a2ac P 2ea47313d10a4705bf36329be4a87410 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:02:34.965328 3979 raft_consensus.cc:2454] T a792ee4c209b4db9a22f42d65729a2ac P 2ea47313d10a4705bf36329be4a87410 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed9f4412136e40cc82447ec2dbda71b9 in term 1.
I20260809 02:02:34.965415 3831 raft_consensus.cc:2454] T a792ee4c209b4db9a22f42d65729a2ac P 3a1dc6be7d1640ba97604a9c62f597e9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed9f4412136e40cc82447ec2dbda71b9 in term 1.
I20260809 02:02:34.965651 4059 leader_election.cc:302] T a792ee4c209b4db9a22f42d65729a2ac P ed9f4412136e40cc82447ec2dbda71b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ea47313d10a4705bf36329be4a87410, ed9f4412136e40cc82447ec2dbda71b9; no voters:
I20260809 02:02:34.965832 4322 raft_consensus.cc:2781] T a792ee4c209b4db9a22f42d65729a2ac P ed9f4412136e40cc82447ec2dbda71b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:02:34.966092 4322 raft_consensus.cc:686] T a792ee4c209b4db9a22f42d65729a2ac P ed9f4412136e40cc82447ec2dbda71b9 [term 1 LEADER]: Becoming Leader. State: Replica: ed9f4412136e40cc82447ec2dbda71b9, State: Running, Role: LEADER
I20260809 02:02:34.966355 4322 consensus_queue.cc:237] T a792ee4c209b4db9a22f42d65729a2ac P ed9f4412136e40cc82447ec2dbda71b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1dc6be7d1640ba97604a9c62f597e9" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 34563 } } peers { permanent_uuid: "2ea47313d10a4705bf36329be4a87410" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38867 } } peers { permanent_uuid: "ed9f4412136e40cc82447ec2dbda71b9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 43843 } }
I20260809 02:02:34.968654 3660 catalog_manager.cc:5495] T a792ee4c209b4db9a22f42d65729a2ac P ed9f4412136e40cc82447ec2dbda71b9 reported cstate change: term changed from 0 to 1, leader changed from <none> to ed9f4412136e40cc82447ec2dbda71b9 (127.1.192.195). New cstate: current_term: 1 leader_uuid: "ed9f4412136e40cc82447ec2dbda71b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1dc6be7d1640ba97604a9c62f597e9" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 34563 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ea47313d10a4705bf36329be4a87410" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38867 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ed9f4412136e40cc82447ec2dbda71b9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 43843 } health_report { overall_health: HEALTHY } } }
I20260809 02:02:34.992930 4332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:34.989246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59144 (local address 127.1.192.195:43843)
0809 02:02:34.989558 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:34.989566 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:34.989580 (+ 14us) server_negotiation.cc:407] Connection header received
0809 02:02:34.989668 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:34.989670 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:34.989723 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:34.989829 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:02:34.990057 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:34.990717 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:34.991544 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:34.991773 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:34.991851 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:34.992112 (+ 261us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:02:34.992583 (+ 471us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:02:34.992672 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:34.992739 (+ 67us) server_negotiation.cc:299] Negotiation successful
0809 02:02:34.992786 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":133,"threads_started":1}
I20260809 02:02:35.003504 4324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:35.000181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.195:41099 (local address 127.1.192.193:34563)
0809 02:02:35.000346 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:35.000351 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:35.000366 (+ 15us) server_negotiation.cc:407] Connection header received
0809 02:02:35.000509 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:35.000519 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:35.000566 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:35.000651 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:02:35.000925 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:35.001437 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:35.002387 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:35.002619 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:35.002733 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:35.003046 (+ 313us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:02:35.003120 (+ 74us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:02:35.003206 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:35.003296 (+ 90us) server_negotiation.cc:299] Negotiation successful
0809 02:02:35.003353 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 02:02:35.003523 4335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:34.999912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.193:34563 (local address 127.1.192.195:41099)
0809 02:02:35.000259 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:35.000277 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 02:02:35.000381 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:02:35.000664 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:02:35.000666 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:02:35.000675 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 02:02:35.000809 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:35.000815 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:35.001588 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:35.001594 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:35.002258 (+ 664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:35.002267 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:35.002744 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:35.002748 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:35.002846 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:35.002891 (+ 45us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 02:02:35.003210 (+ 319us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 02:02:35.003220 (+ 10us) client_negotiation.cc:769] Sending connection context
0809 02:02:35.003287 (+ 67us) client_negotiation.cc:240] Negotiation successful
0809 02:02:35.003370 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":128,"threads_started":1}
I20260809 02:02:35.010402 3979 raft_consensus.cc:1261] T a792ee4c209b4db9a22f42d65729a2ac P 2ea47313d10a4705bf36329be4a87410 [term 1 FOLLOWER]: Refusing update from remote peer ed9f4412136e40cc82447ec2dbda71b9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:02:35.010505 3831 raft_consensus.cc:1261] T a792ee4c209b4db9a22f42d65729a2ac P 3a1dc6be7d1640ba97604a9c62f597e9 [term 1 FOLLOWER]: Refusing update from remote peer ed9f4412136e40cc82447ec2dbda71b9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:02:35.010912 4322 consensus_queue.cc:1035] T a792ee4c209b4db9a22f42d65729a2ac P ed9f4412136e40cc82447ec2dbda71b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ea47313d10a4705bf36329be4a87410" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38867 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:02:35.011152 4328 consensus_queue.cc:1035] T a792ee4c209b4db9a22f42d65729a2ac P ed9f4412136e40cc82447ec2dbda71b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a1dc6be7d1640ba97604a9c62f597e9" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 34563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:02:35.029541 4332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:35.026126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.193:55971 (local address 127.1.192.195:43843)
0809 02:02:35.026250 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:35.026254 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:35.026517 (+ 263us) server_negotiation.cc:407] Connection header received
0809 02:02:35.026631 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:35.026634 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:35.026698 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:35.026786 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:02:35.027036 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:35.027553 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:35.028423 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:35.028676 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:35.028791 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:35.029057 (+ 266us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:02:35.029137 (+ 80us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:02:35.029220 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:35.029311 (+ 91us) server_negotiation.cc:299] Negotiation successful
0809 02:02:35.029365 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260809 02:02:35.029546 4340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:35.026061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.195:43843 (local address 127.1.192.193:55971)
0809 02:02:35.026380 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:35.026399 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 02:02:35.026532 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:02:35.026801 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:02:35.026804 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:02:35.026813 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 02:02:35.026924 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:35.026932 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:35.027673 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:35.027680 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:35.028297 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:35.028305 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:35.028795 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:35.028798 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:35.028889 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:35.028925 (+ 36us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 02:02:35.029226 (+ 301us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 02:02:35.029235 (+ 9us) client_negotiation.cc:769] Sending connection context
0809 02:02:35.029301 (+ 66us) client_negotiation.cc:240] Negotiation successful
0809 02:02:35.029388 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":152,"threads_started":1}
I20260809 02:02:35.054324 3660 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36344:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:02:35.054525 3660 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36344:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:02:35.056888 3660 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5731d70307644f7ea1a3ccabcb93f4c3: Sending DeleteTablet for 3 replicas of tablet a792ee4c209b4db9a22f42d65729a2ac
I20260809 02:02:35.057710 4113 tablet_service.cc:1514] Processing DeleteTablet for tablet a792ee4c209b4db9a22f42d65729a2ac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:02:35 UTC) from {username='kudu', principal='kudu/127.1.192.254@KRBTEST.COM'} at 127.0.0.1:59128
I20260809 02:02:35.057687 3958 tablet_service.cc:1514] Processing DeleteTablet for tablet a792ee4c209b4db9a22f42d65729a2ac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:02:35 UTC) from {username='kudu', principal='kudu/127.1.192.254@KRBTEST.COM'} at 127.0.0.1:54768
I20260809 02:02:35.057695 3810 tablet_service.cc:1514] Processing DeleteTablet for tablet a792ee4c209b4db9a22f42d65729a2ac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:02:35 UTC) from {username='kudu', principal='kudu/127.1.192.254@KRBTEST.COM'} at 127.0.0.1:48422
I20260809 02:02:35.057899 4304 tablet_replica.cc:330] T a792ee4c209b4db9a22f42d65729a2ac P 2ea47313d10a4705bf36329be4a87410: stopping tablet replica
I20260809 02:02:35.057921 4306 tablet_replica.cc:330] T a792ee4c209b4db9a22f42d65729a2ac P ed9f4412136e40cc82447ec2dbda71b9: stopping tablet replica
I20260809 02:02:35.058029 4304 raft_consensus.cc:2229] T a792ee4c209b4db9a22f42d65729a2ac P 2ea47313d10a4705bf36329be4a87410 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:02:35.058045 4306 raft_consensus.cc:2229] T a792ee4c209b4db9a22f42d65729a2ac P ed9f4412136e40cc82447ec2dbda71b9 [term 1 LEADER]: Raft consensus shutting down.
I20260809 02:02:35.058166 4304 raft_consensus.cc:2258] T a792ee4c209b4db9a22f42d65729a2ac P 2ea47313d10a4705bf36329be4a87410 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:02:35.058373 4306 raft_consensus.cc:2258] T a792ee4c209b4db9a22f42d65729a2ac P ed9f4412136e40cc82447ec2dbda71b9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:02:35.058597 4305 tablet_replica.cc:330] T a792ee4c209b4db9a22f42d65729a2ac P 3a1dc6be7d1640ba97604a9c62f597e9: stopping tablet replica
I20260809 02:02:35.058749 4305 raft_consensus.cc:2229] T a792ee4c209b4db9a22f42d65729a2ac P 3a1dc6be7d1640ba97604a9c62f597e9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:02:35.058874 4305 raft_consensus.cc:2258] T a792ee4c209b4db9a22f42d65729a2ac P 3a1dc6be7d1640ba97604a9c62f597e9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:02:35.059410 3660 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36344:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 02:02:35.059569 4304 ts_tablet_manager.cc:1905] T a792ee4c209b4db9a22f42d65729a2ac P 2ea47313d10a4705bf36329be4a87410: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:02:35.059751 4305 ts_tablet_manager.cc:1905] T a792ee4c209b4db9a22f42d65729a2ac P 3a1dc6be7d1640ba97604a9c62f597e9: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 02:02:35.060129 3660 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:02:35.062422 4304 ts_tablet_manager.cc:1918] T a792ee4c209b4db9a22f42d65729a2ac P 2ea47313d10a4705bf36329be4a87410: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 02:02:35.062503 4305 ts_tablet_manager.cc:1918] T a792ee4c209b4db9a22f42d65729a2ac P 3a1dc6be7d1640ba97604a9c62f597e9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 02:02:35.062528 4304 log.cc:1196] T a792ee4c209b4db9a22f42d65729a2ac P 2ea47313d10a4705bf36329be4a87410: Deleting WAL directory at /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786240941968412-1795-0/minicluster-data/ts-1/wal/wals/a792ee4c209b4db9a22f42d65729a2ac
I20260809 02:02:35.062597 4305 log.cc:1196] T a792ee4c209b4db9a22f42d65729a2ac P 3a1dc6be7d1640ba97604a9c62f597e9: Deleting WAL directory at /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786240941968412-1795-0/minicluster-data/ts-0/wal/wals/a792ee4c209b4db9a22f42d65729a2ac
I20260809 02:02:35.062831 4304 ts_tablet_manager.cc:1939] T a792ee4c209b4db9a22f42d65729a2ac P 2ea47313d10a4705bf36329be4a87410: Deleting consensus metadata
I20260809 02:02:35.062908 4305 ts_tablet_manager.cc:1939] T a792ee4c209b4db9a22f42d65729a2ac P 3a1dc6be7d1640ba97604a9c62f597e9: Deleting consensus metadata
I20260809 02:02:35.063606 3645 catalog_manager.cc:4841] TS 3a1dc6be7d1640ba97604a9c62f597e9 (127.1.192.193:34563): tablet a792ee4c209b4db9a22f42d65729a2ac (table test-table [id=507b9ffef78741c1854c6981641cca46]) successfully deleted
I20260809 02:02:35.063772 3645 catalog_manager.cc:4841] TS 2ea47313d10a4705bf36329be4a87410 (127.1.192.194:38867): tablet a792ee4c209b4db9a22f42d65729a2ac (table test-table [id=507b9ffef78741c1854c6981641cca46]) successfully deleted
I20260809 02:02:35.064131 4306 ts_tablet_manager.cc:1905] T a792ee4c209b4db9a22f42d65729a2ac P ed9f4412136e40cc82447ec2dbda71b9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:02:35.066778 4306 ts_tablet_manager.cc:1918] T a792ee4c209b4db9a22f42d65729a2ac P ed9f4412136e40cc82447ec2dbda71b9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 02:02:35.066864 4306 log.cc:1196] T a792ee4c209b4db9a22f42d65729a2ac P ed9f4412136e40cc82447ec2dbda71b9: Deleting WAL directory at /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786240941968412-1795-0/minicluster-data/ts-2/wal/wals/a792ee4c209b4db9a22f42d65729a2ac
I20260809 02:02:35.067091 3958 tablet_service.cc:1467] Processing CreateTablet for tablet b6c14ee5f7a44310b303a133e56f8bc1 (DEFAULT_TABLE table=test-table [id=dc4e738ab7ff44079394da14e3f4bda5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:02:35.067245 3810 tablet_service.cc:1467] Processing CreateTablet for tablet b6c14ee5f7a44310b303a133e56f8bc1 (DEFAULT_TABLE table=test-table [id=dc4e738ab7ff44079394da14e3f4bda5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:02:35.067435 3958 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b6c14ee5f7a44310b303a133e56f8bc1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:02:35.067505 3810 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b6c14ee5f7a44310b303a133e56f8bc1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:02:35.069727 4316 tablet_bootstrap.cc:492] T b6c14ee5f7a44310b303a133e56f8bc1 P 2ea47313d10a4705bf36329be4a87410: Bootstrap starting.
I20260809 02:02:35.069963 4318 tablet_bootstrap.cc:492] T b6c14ee5f7a44310b303a133e56f8bc1 P 3a1dc6be7d1640ba97604a9c62f597e9: Bootstrap starting.
I20260809 02:02:35.070735 4316 tablet_bootstrap.cc:654] T b6c14ee5f7a44310b303a133e56f8bc1 P 2ea47313d10a4705bf36329be4a87410: Neither blocks nor log segments found. Creating new log.
I20260809 02:02:35.070883 4318 tablet_bootstrap.cc:654] T b6c14ee5f7a44310b303a133e56f8bc1 P 3a1dc6be7d1640ba97604a9c62f597e9: Neither blocks nor log segments found. Creating new log.
I20260809 02:02:35.071587 4113 tablet_service.cc:1467] Processing CreateTablet for tablet b6c14ee5f7a44310b303a133e56f8bc1 (DEFAULT_TABLE table=test-table [id=dc4e738ab7ff44079394da14e3f4bda5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:02:35.071709 4306 ts_tablet_manager.cc:1939] T a792ee4c209b4db9a22f42d65729a2ac P ed9f4412136e40cc82447ec2dbda71b9: Deleting consensus metadata
I20260809 02:02:35.071820 4113 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b6c14ee5f7a44310b303a133e56f8bc1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:02:35.072458 3643 catalog_manager.cc:4841] TS ed9f4412136e40cc82447ec2dbda71b9 (127.1.192.195:43843): tablet a792ee4c209b4db9a22f42d65729a2ac (table test-table [id=507b9ffef78741c1854c6981641cca46]) successfully deleted
I20260809 02:02:35.073086 4318 tablet_bootstrap.cc:492] T b6c14ee5f7a44310b303a133e56f8bc1 P 3a1dc6be7d1640ba97604a9c62f597e9: No bootstrap required, opened a new log
I20260809 02:02:35.073174 4318 ts_tablet_manager.cc:1397] T b6c14ee5f7a44310b303a133e56f8bc1 P 3a1dc6be7d1640ba97604a9c62f597e9: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260809 02:02:35.073616 4318 raft_consensus.cc:348] T b6c14ee5f7a44310b303a133e56f8bc1 P 3a1dc6be7d1640ba97604a9c62f597e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1dc6be7d1640ba97604a9c62f597e9" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 34563 } } peers { permanent_uuid: "2ea47313d10a4705bf36329be4a87410" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38867 } } peers { permanent_uuid: "ed9f4412136e40cc82447ec2dbda71b9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 43843 } }
I20260809 02:02:35.073752 4318 raft_consensus.cc:374] T b6c14ee5f7a44310b303a133e56f8bc1 P 3a1dc6be7d1640ba97604a9c62f597e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:02:35.073791 4318 raft_consensus.cc:729] T b6c14ee5f7a44310b303a133e56f8bc1 P 3a1dc6be7d1640ba97604a9c62f597e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a1dc6be7d1640ba97604a9c62f597e9, State: Initialized, Role: FOLLOWER
I20260809 02:02:35.073927 4318 consensus_queue.cc:260] T b6c14ee5f7a44310b303a133e56f8bc1 P 3a1dc6be7d1640ba97604a9c62f597e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1dc6be7d1640ba97604a9c62f597e9" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 34563 } } peers { permanent_uuid: "2ea47313d10a4705bf36329be4a87410" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38867 } } peers { permanent_uuid: "ed9f4412136e40cc82447ec2dbda71b9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 43843 } }
I20260809 02:02:35.074023 4316 tablet_bootstrap.cc:492] T b6c14ee5f7a44310b303a133e56f8bc1 P 2ea47313d10a4705bf36329be4a87410: No bootstrap required, opened a new log
I20260809 02:02:35.074100 4316 ts_tablet_manager.cc:1397] T b6c14ee5f7a44310b303a133e56f8bc1 P 2ea47313d10a4705bf36329be4a87410: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260809 02:02:35.074208 4318 ts_tablet_manager.cc:1428] T b6c14ee5f7a44310b303a133e56f8bc1 P 3a1dc6be7d1640ba97604a9c62f597e9: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 02:02:35.074252 4317 tablet_bootstrap.cc:492] T b6c14ee5f7a44310b303a133e56f8bc1 P ed9f4412136e40cc82447ec2dbda71b9: Bootstrap starting.
I20260809 02:02:35.074462 4316 raft_consensus.cc:348] T b6c14ee5f7a44310b303a133e56f8bc1 P 2ea47313d10a4705bf36329be4a87410 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1dc6be7d1640ba97604a9c62f597e9" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 34563 } } peers { permanent_uuid: "2ea47313d10a4705bf36329be4a87410" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38867 } } peers { permanent_uuid: "ed9f4412136e40cc82447ec2dbda71b9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 43843 } }
I20260809 02:02:35.074560 4316 raft_consensus.cc:374] T b6c14ee5f7a44310b303a133e56f8bc1 P 2ea47313d10a4705bf36329be4a87410 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:02:35.074602 4316 raft_consensus.cc:729] T b6c14ee5f7a44310b303a133e56f8bc1 P 2ea47313d10a4705bf36329be4a87410 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ea47313d10a4705bf36329be4a87410, State: Initialized, Role: FOLLOWER
I20260809 02:02:35.074716 4316 consensus_queue.cc:260] T b6c14ee5f7a44310b303a133e56f8bc1 P 2ea47313d10a4705bf36329be4a87410 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1dc6be7d1640ba97604a9c62f597e9" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 34563 } } peers { permanent_uuid: "2ea47313d10a4705bf36329be4a87410" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38867 } } peers { permanent_uuid: "ed9f4412136e40cc82447ec2dbda71b9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 43843 } }
I20260809 02:02:35.074949 4316 ts_tablet_manager.cc:1428] T b6c14ee5f7a44310b303a133e56f8bc1 P 2ea47313d10a4705bf36329be4a87410: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 02:02:35.075289 4317 tablet_bootstrap.cc:654] T b6c14ee5f7a44310b303a133e56f8bc1 P ed9f4412136e40cc82447ec2dbda71b9: Neither blocks nor log segments found. Creating new log.
I20260809 02:02:35.077054 4317 tablet_bootstrap.cc:492] T b6c14ee5f7a44310b303a133e56f8bc1 P ed9f4412136e40cc82447ec2dbda71b9: No bootstrap required, opened a new log
I20260809 02:02:35.077147 4317 ts_tablet_manager.cc:1397] T b6c14ee5f7a44310b303a133e56f8bc1 P ed9f4412136e40cc82447ec2dbda71b9: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 02:02:35.077649 4317 raft_consensus.cc:348] T b6c14ee5f7a44310b303a133e56f8bc1 P ed9f4412136e40cc82447ec2dbda71b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1dc6be7d1640ba97604a9c62f597e9" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 34563 } } peers { permanent_uuid: "2ea47313d10a4705bf36329be4a87410" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38867 } } peers { permanent_uuid: "ed9f4412136e40cc82447ec2dbda71b9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 43843 } }
I20260809 02:02:35.077781 4317 raft_consensus.cc:374] T b6c14ee5f7a44310b303a133e56f8bc1 P ed9f4412136e40cc82447ec2dbda71b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:02:35.077824 4317 raft_consensus.cc:729] T b6c14ee5f7a44310b303a133e56f8bc1 P ed9f4412136e40cc82447ec2dbda71b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed9f4412136e40cc82447ec2dbda71b9, State: Initialized, Role: FOLLOWER
I20260809 02:02:35.077943 4317 consensus_queue.cc:260] T b6c14ee5f7a44310b303a133e56f8bc1 P ed9f4412136e40cc82447ec2dbda71b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1dc6be7d1640ba97604a9c62f597e9" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 34563 } } peers { permanent_uuid: "2ea47313d10a4705bf36329be4a87410" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38867 } } peers { permanent_uuid: "ed9f4412136e40cc82447ec2dbda71b9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 43843 } }
I20260809 02:02:35.078222 4317 ts_tablet_manager.cc:1428] T b6c14ee5f7a44310b303a133e56f8bc1 P ed9f4412136e40cc82447ec2dbda71b9: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 02:02:35.241578 4223 raft_consensus.cc:482] T b6c14ee5f7a44310b303a133e56f8bc1 P 3a1dc6be7d1640ba97604a9c62f597e9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:02:35.241748 4223 raft_consensus.cc:504] T b6c14ee5f7a44310b303a133e56f8bc1 P 3a1dc6be7d1640ba97604a9c62f597e9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1dc6be7d1640ba97604a9c62f597e9" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 34563 } } peers { permanent_uuid: "2ea47313d10a4705bf36329be4a87410" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38867 } } peers { permanent_uuid: "ed9f4412136e40cc82447ec2dbda71b9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 43843 } }
I20260809 02:02:35.242218 4223 leader_election.cc:288] T b6c14ee5f7a44310b303a133e56f8bc1 P 3a1dc6be7d1640ba97604a9c62f597e9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2ea47313d10a4705bf36329be4a87410 (127.1.192.194:38867), ed9f4412136e40cc82447ec2dbda71b9 (127.1.192.195:43843)
I20260809 02:02:35.242760 3979 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6c14ee5f7a44310b303a133e56f8bc1" candidate_uuid: "3a1dc6be7d1640ba97604a9c62f597e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ea47313d10a4705bf36329be4a87410" is_pre_election: true
I20260809 02:02:35.242760 4135 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6c14ee5f7a44310b303a133e56f8bc1" candidate_uuid: "3a1dc6be7d1640ba97604a9c62f597e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed9f4412136e40cc82447ec2dbda71b9" is_pre_election: true
I20260809 02:02:35.242937 4135 raft_consensus.cc:2454] T b6c14ee5f7a44310b303a133e56f8bc1 P ed9f4412136e40cc82447ec2dbda71b9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a1dc6be7d1640ba97604a9c62f597e9 in term 0.
I20260809 02:02:35.242937 3979 raft_consensus.cc:2454] T b6c14ee5f7a44310b303a133e56f8bc1 P 2ea47313d10a4705bf36329be4a87410 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a1dc6be7d1640ba97604a9c62f597e9 in term 0.
I20260809 02:02:35.243341 3756 leader_election.cc:302] T b6c14ee5f7a44310b303a133e56f8bc1 P 3a1dc6be7d1640ba97604a9c62f597e9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a1dc6be7d1640ba97604a9c62f597e9, ed9f4412136e40cc82447ec2dbda71b9; no voters:
I20260809 02:02:35.243603 4223 raft_consensus.cc:2781] T b6c14ee5f7a44310b303a133e56f8bc1 P 3a1dc6be7d1640ba97604a9c62f597e9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:02:35.243682 4223 raft_consensus.cc:482] T b6c14ee5f7a44310b303a133e56f8bc1 P 3a1dc6be7d1640ba97604a9c62f597e9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:02:35.243711 4223 raft_consensus.cc:3037] T b6c14ee5f7a44310b303a133e56f8bc1 P 3a1dc6be7d1640ba97604a9c62f597e9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:02:35.244730 4223 raft_consensus.cc:504] T b6c14ee5f7a44310b303a133e56f8bc1 P 3a1dc6be7d1640ba97604a9c62f597e9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1dc6be7d1640ba97604a9c62f597e9" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 34563 } } peers { permanent_uuid: "2ea47313d10a4705bf36329be4a87410" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38867 } } peers { permanent_uuid: "ed9f4412136e40cc82447ec2dbda71b9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 43843 } }
I20260809 02:02:35.245113 4223 leader_election.cc:288] T b6c14ee5f7a44310b303a133e56f8bc1 P 3a1dc6be7d1640ba97604a9c62f597e9 [CANDIDATE]: Term 1 election: Requested vote from peers 2ea47313d10a4705bf36329be4a87410 (127.1.192.194:38867), ed9f4412136e40cc82447ec2dbda71b9 (127.1.192.195:43843)
I20260809 02:02:35.245524 3979 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6c14ee5f7a44310b303a133e56f8bc1" candidate_uuid: "3a1dc6be7d1640ba97604a9c62f597e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ea47313d10a4705bf36329be4a87410"
I20260809 02:02:35.245540 4135 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6c14ee5f7a44310b303a133e56f8bc1" candidate_uuid: "3a1dc6be7d1640ba97604a9c62f597e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed9f4412136e40cc82447ec2dbda71b9"
I20260809 02:02:35.245671 4135 raft_consensus.cc:3037] T b6c14ee5f7a44310b303a133e56f8bc1 P ed9f4412136e40cc82447ec2dbda71b9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:02:35.245671 3979 raft_consensus.cc:3037] T b6c14ee5f7a44310b303a133e56f8bc1 P 2ea47313d10a4705bf36329be4a87410 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:02:35.246994 3979 raft_consensus.cc:2454] T b6c14ee5f7a44310b303a133e56f8bc1 P 2ea47313d10a4705bf36329be4a87410 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a1dc6be7d1640ba97604a9c62f597e9 in term 1.
I20260809 02:02:35.246995 4135 raft_consensus.cc:2454] T b6c14ee5f7a44310b303a133e56f8bc1 P ed9f4412136e40cc82447ec2dbda71b9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a1dc6be7d1640ba97604a9c62f597e9 in term 1.
I20260809 02:02:35.247351 3756 leader_election.cc:302] T b6c14ee5f7a44310b303a133e56f8bc1 P 3a1dc6be7d1640ba97604a9c62f597e9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a1dc6be7d1640ba97604a9c62f597e9, ed9f4412136e40cc82447ec2dbda71b9; no voters:
I20260809 02:02:35.247547 4223 raft_consensus.cc:2781] T b6c14ee5f7a44310b303a133e56f8bc1 P 3a1dc6be7d1640ba97604a9c62f597e9 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:02:35.247642 4223 raft_consensus.cc:686] T b6c14ee5f7a44310b303a133e56f8bc1 P 3a1dc6be7d1640ba97604a9c62f597e9 [term 1 LEADER]: Becoming Leader. State: Replica: 3a1dc6be7d1640ba97604a9c62f597e9, State: Running, Role: LEADER
I20260809 02:02:35.247788 4223 consensus_queue.cc:237] T b6c14ee5f7a44310b303a133e56f8bc1 P 3a1dc6be7d1640ba97604a9c62f597e9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1dc6be7d1640ba97604a9c62f597e9" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 34563 } } peers { permanent_uuid: "2ea47313d10a4705bf36329be4a87410" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38867 } } peers { permanent_uuid: "ed9f4412136e40cc82447ec2dbda71b9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 43843 } }
I20260809 02:02:35.249681 3659 catalog_manager.cc:5495] T b6c14ee5f7a44310b303a133e56f8bc1 P 3a1dc6be7d1640ba97604a9c62f597e9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3a1dc6be7d1640ba97604a9c62f597e9 (127.1.192.193). New cstate: current_term: 1 leader_uuid: "3a1dc6be7d1640ba97604a9c62f597e9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1dc6be7d1640ba97604a9c62f597e9" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 34563 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2ea47313d10a4705bf36329be4a87410" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38867 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ed9f4412136e40cc82447ec2dbda71b9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 43843 } health_report { overall_health: UNKNOWN } } }
I20260809 02:02:35.277825 4324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:35.274506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48444 (local address 127.1.192.193:34563)
0809 02:02:35.274628 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:35.274633 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:35.274709 (+ 76us) server_negotiation.cc:407] Connection header received
0809 02:02:35.274869 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:35.274877 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:35.274923 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:35.275004 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:02:35.275278 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:35.275867 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:35.276734 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:35.276954 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:35.277069 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:35.277336 (+ 267us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:02:35.277409 (+ 73us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:02:35.277489 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:35.277616 (+ 127us) server_negotiation.cc:299] Negotiation successful
0809 02:02:35.277674 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 02:02:35.285208 3979 raft_consensus.cc:1261] T b6c14ee5f7a44310b303a133e56f8bc1 P 2ea47313d10a4705bf36329be4a87410 [term 1 FOLLOWER]: Refusing update from remote peer 3a1dc6be7d1640ba97604a9c62f597e9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:02:35.285323 4135 raft_consensus.cc:1261] T b6c14ee5f7a44310b303a133e56f8bc1 P ed9f4412136e40cc82447ec2dbda71b9 [term 1 FOLLOWER]: Refusing update from remote peer 3a1dc6be7d1640ba97604a9c62f597e9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:02:35.285655 4223 consensus_queue.cc:1035] T b6c14ee5f7a44310b303a133e56f8bc1 P 3a1dc6be7d1640ba97604a9c62f597e9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ea47313d10a4705bf36329be4a87410" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38867 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:02:35.285820 4284 consensus_queue.cc:1035] T b6c14ee5f7a44310b303a133e56f8bc1 P 3a1dc6be7d1640ba97604a9c62f597e9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed9f4412136e40cc82447ec2dbda71b9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 43843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 02:02:35.299722 4282 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260809 02:02:36.863756 3660 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36344:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:02:36.863938 3660 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36344:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:02:36.866726 3660 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5731d70307644f7ea1a3ccabcb93f4c3: Sending DeleteTablet for 3 replicas of tablet b6c14ee5f7a44310b303a133e56f8bc1
I20260809 02:02:36.867354 3810 tablet_service.cc:1514] Processing DeleteTablet for tablet b6c14ee5f7a44310b303a133e56f8bc1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:02:36 UTC) from {username='kudu', principal='kudu/127.1.192.254@KRBTEST.COM'} at 127.0.0.1:48422
I20260809 02:02:36.867434 4113 tablet_service.cc:1514] Processing DeleteTablet for tablet b6c14ee5f7a44310b303a133e56f8bc1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:02:36 UTC) from {username='kudu', principal='kudu/127.1.192.254@KRBTEST.COM'} at 127.0.0.1:59128
I20260809 02:02:36.867795 3958 tablet_service.cc:1514] Processing DeleteTablet for tablet b6c14ee5f7a44310b303a133e56f8bc1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:02:36 UTC) from {username='kudu', principal='kudu/127.1.192.254@KRBTEST.COM'} at 127.0.0.1:54768
I20260809 02:02:36.867827 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 3738
I20260809 02:02:36.867847 4372 tablet_replica.cc:330] T b6c14ee5f7a44310b303a133e56f8bc1 P 3a1dc6be7d1640ba97604a9c62f597e9: stopping tablet replica
I20260809 02:02:36.867899 4373 tablet_replica.cc:330] T b6c14ee5f7a44310b303a133e56f8bc1 P ed9f4412136e40cc82447ec2dbda71b9: stopping tablet replica
I20260809 02:02:36.869761 4373 raft_consensus.cc:2229] T b6c14ee5f7a44310b303a133e56f8bc1 P ed9f4412136e40cc82447ec2dbda71b9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:02:36.869906 4373 raft_consensus.cc:2258] T b6c14ee5f7a44310b303a133e56f8bc1 P ed9f4412136e40cc82447ec2dbda71b9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:02:36.870474 4374 tablet_replica.cc:330] T b6c14ee5f7a44310b303a133e56f8bc1 P 2ea47313d10a4705bf36329be4a87410: stopping tablet replica
I20260809 02:02:36.870654 4374 raft_consensus.cc:2229] T b6c14ee5f7a44310b303a133e56f8bc1 P 2ea47313d10a4705bf36329be4a87410 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:02:36.870805 4374 raft_consensus.cc:2258] T b6c14ee5f7a44310b303a133e56f8bc1 P 2ea47313d10a4705bf36329be4a87410 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:02:36.871611 4374 ts_tablet_manager.cc:1905] T b6c14ee5f7a44310b303a133e56f8bc1 P 2ea47313d10a4705bf36329be4a87410: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:02:36.871747 4373 ts_tablet_manager.cc:1905] T b6c14ee5f7a44310b303a133e56f8bc1 P ed9f4412136e40cc82447ec2dbda71b9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:02:36.874513 4374 ts_tablet_manager.cc:1918] T b6c14ee5f7a44310b303a133e56f8bc1 P 2ea47313d10a4705bf36329be4a87410: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 02:02:36.874513 4373 ts_tablet_manager.cc:1918] T b6c14ee5f7a44310b303a133e56f8bc1 P ed9f4412136e40cc82447ec2dbda71b9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 02:02:36.874612 4373 log.cc:1196] T b6c14ee5f7a44310b303a133e56f8bc1 P ed9f4412136e40cc82447ec2dbda71b9: Deleting WAL directory at /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786240941968412-1795-0/minicluster-data/ts-2/wal/wals/b6c14ee5f7a44310b303a133e56f8bc1
I20260809 02:02:36.874680 4374 log.cc:1196] T b6c14ee5f7a44310b303a133e56f8bc1 P 2ea47313d10a4705bf36329be4a87410: Deleting WAL directory at /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786240941968412-1795-0/minicluster-data/ts-1/wal/wals/b6c14ee5f7a44310b303a133e56f8bc1
I20260809 02:02:36.874914 4373 ts_tablet_manager.cc:1939] T b6c14ee5f7a44310b303a133e56f8bc1 P ed9f4412136e40cc82447ec2dbda71b9: Deleting consensus metadata
I20260809 02:02:36.874980 4374 ts_tablet_manager.cc:1939] T b6c14ee5f7a44310b303a133e56f8bc1 P 2ea47313d10a4705bf36329be4a87410: Deleting consensus metadata
I20260809 02:02:36.876086 3645 catalog_manager.cc:4841] TS 2ea47313d10a4705bf36329be4a87410 (127.1.192.194:38867): tablet b6c14ee5f7a44310b303a133e56f8bc1 (table test-table [id=dc4e738ab7ff44079394da14e3f4bda5]) successfully deleted
I20260809 02:02:36.877952 3643 catalog_manager.cc:4841] TS ed9f4412136e40cc82447ec2dbda71b9 (127.1.192.195:43843): tablet b6c14ee5f7a44310b303a133e56f8bc1 (table test-table [id=dc4e738ab7ff44079394da14e3f4bda5]) successfully deleted
W20260809 02:02:36.880491 3645 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.192.193:34563 (error 108)
I20260809 02:02:36.880828 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 3889
I20260809 02:02:36.883193 4376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:36.882616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.193:34563 (local address 127.0.0.1:48456)
0809 02:02:36.882951 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:36.883046 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.193:34563: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":80,"threads_started":1}
W20260809 02:02:36.883347 3645 catalog_manager.cc:4584] TS 3a1dc6be7d1640ba97604a9c62f597e9 (127.1.192.193:34563): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b6c14ee5f7a44310b303a133e56f8bc1: Network error: Client connection negotiation failed: client connection to 127.1.192.193:34563: connect: Connection refused (error 111)
I20260809 02:02:36.891232 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 4038
I20260809 02:02:36.900957 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 3627
2026-08-09T02:02:36Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6463 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786240941968412-1795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:02:36 dist-test-slave-ss4n krb5kdc[4379](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:02:36 dist-test-slave-ss4n krb5kdc[4379](info): set up 2 sockets
Aug 09 02:02:36 dist-test-slave-ss4n krb5kdc[4379](info): commencing operation
krb5kdc: starting...
W20260809 02:02:38.035807 1795 mini_kdc.cc:121] Time spent starting KDC: real 1.108s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786240941968412-1795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786240941968412-1795-0/krb5kdc/test-user.keytab.
Aug 09 02:02:38 dist-test-slave-ss4n krb5kdc[4379](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240958, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:02:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:02:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.192.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.192.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.192.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab.
Entry for principal kudu/127.1.192.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab.
I20260809 02:02:38.169912 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786240941968412-1795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786240941968412-1795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786240941968412-1795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.192.254:38497
--webserver_interface=127.1.192.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab
--principal=kudu/127.1.192.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:46231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.192.254:38497
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:02:38.284520 4393 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:02:38.284880 4393 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:02:38.284989 4393 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:02:38.289196 4393 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:02:38.289307 4393 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:02:38.289347 4393 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:02:38.289405 4393 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:02:38.289443 4393 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:02:38.295176 4393 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:46231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786240941968412-1795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786240941968412-1795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.192.254:38497
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab
--principal=kudu/127.1.192.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.192.254:38497
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.192.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4393
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786240941968412-1795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:02:38.296389 4393 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:02:38.297470 4393 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:02:38.306209 4399 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:02:38.306246 4398 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:02:38.306236 4401 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:02:38.306654 4393 server_base.cc:1034] running on GCE node
I20260809 02:02:38.307299 4393 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:02:38.308495 4393 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:02:38.309667 4393 hybrid_clock.cc:648] HybridClock initialized: now 1786240958309638 us; error 42 us; skew 500 ppm
Aug 09 02:02:38 dist-test-slave-ss4n krb5kdc[4379](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240958, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:02:38.312636 4393 init.cc:377] Logged in from keytab as kudu/127.1.192.254@KRBTEST.COM (short username kudu)
I20260809 02:02:38.313666 4393 webserver.cc:460] Webserver started at http://127.1.192.254:44387/ using document root <none> and password file <none>
I20260809 02:02:38.314297 4393 fs_manager.cc:359] Metadata directory not provided
I20260809 02:02:38.314399 4393 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:02:38.314640 4393 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:02:38.316937 4393 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786240941968412-1795-0/minicluster-data/master-0/data/instance:
uuid: "d95894a8c1174b9d80a0f01039daafbb"
format_stamp: "Formatted at 2026-08-09 02:02:38 on dist-test-slave-ss4n"
server_key: "3439e7b2d3bd3b8ee1b1d5c6045f184b"
server_key_iv: "76ff5013ad94ae0d66c2913b124b1a9e"
server_key_version: "encryptionkey@0"
I20260809 02:02:38.317619 4393 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786240941968412-1795-0/minicluster-data/master-0/wal/instance:
uuid: "d95894a8c1174b9d80a0f01039daafbb"
format_stamp: "Formatted at 2026-08-09 02:02:38 on dist-test-slave-ss4n"
server_key: "3439e7b2d3bd3b8ee1b1d5c6045f184b"
server_key_iv: "76ff5013ad94ae0d66c2913b124b1a9e"
server_key_version: "encryptionkey@0"
I20260809 02:02:38.321640 4393 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260809 02:02:38.324220 4416 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:38.325446 4393 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 02:02:38.325636 4393 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786240941968412-1795-0/minicluster-data/master-0/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786240941968412-1795-0/minicluster-data/master-0/wal
uuid: "d95894a8c1174b9d80a0f01039daafbb"
format_stamp: "Formatted at 2026-08-09 02:02:38 on dist-test-slave-ss4n"
server_key: "3439e7b2d3bd3b8ee1b1d5c6045f184b"
server_key_iv: "76ff5013ad94ae0d66c2913b124b1a9e"
server_key_version: "encryptionkey@0"
I20260809 02:02:38.325785 4393 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786240941968412-1795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786240941968412-1795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786240941968412-1795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:02:38.337375 4393 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:02:38.340407 4393 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:02:38.340636 4393 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:02:38.348402 4393 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.254:38497
I20260809 02:02:38.348431 4472 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.254:38497 every 8 connection(s)
I20260809 02:02:38.349432 4393 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
I20260809 02:02:38.352454 4473 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:02:38.357676 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 4393
I20260809 02:02:38.357816 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786240941968412-1795-0/minicluster-data/master-0/wal/instance
I20260809 02:02:38.358094 1795 external_mini_cluster.cc:1428] Setting key 1e13cd98f99711a4cb9bffec2e753261
I20260809 02:02:38.358880 4473 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d95894a8c1174b9d80a0f01039daafbb: Bootstrap starting.
I20260809 02:02:38.361456 4473 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d95894a8c1174b9d80a0f01039daafbb: Neither blocks nor log segments found. Creating new log.
I20260809 02:02:38.362536 4473 log.cc:824] T 00000000000000000000000000000000 P d95894a8c1174b9d80a0f01039daafbb: Log is configured to *not* fsync() on all Append() calls
I20260809 02:02:38.364812 4473 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d95894a8c1174b9d80a0f01039daafbb: No bootstrap required, opened a new log
Aug 09 02:02:38 dist-test-slave-ss4n krb5kdc[4379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240958, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.192.254@KRBTEST.COM
I20260809 02:02:38.369709 4473 raft_consensus.cc:348] T 00000000000000000000000000000000 P d95894a8c1174b9d80a0f01039daafbb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d95894a8c1174b9d80a0f01039daafbb" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 38497 } }
I20260809 02:02:38.370011 4473 raft_consensus.cc:374] T 00000000000000000000000000000000 P d95894a8c1174b9d80a0f01039daafbb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:02:38.370101 4473 raft_consensus.cc:729] T 00000000000000000000000000000000 P d95894a8c1174b9d80a0f01039daafbb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d95894a8c1174b9d80a0f01039daafbb, State: Initialized, Role: FOLLOWER
I20260809 02:02:38.370579 4473 consensus_queue.cc:260] T 00000000000000000000000000000000 P d95894a8c1174b9d80a0f01039daafbb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d95894a8c1174b9d80a0f01039daafbb" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 38497 } }
I20260809 02:02:38.370744 4473 raft_consensus.cc:388] T 00000000000000000000000000000000 P d95894a8c1174b9d80a0f01039daafbb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:02:38.370838 4473 raft_consensus.cc:482] T 00000000000000000000000000000000 P d95894a8c1174b9d80a0f01039daafbb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:02:38.370961 4473 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d95894a8c1174b9d80a0f01039daafbb [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:02:38.371969 4473 raft_consensus.cc:504] T 00000000000000000000000000000000 P d95894a8c1174b9d80a0f01039daafbb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d95894a8c1174b9d80a0f01039daafbb" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 38497 } }
I20260809 02:02:38.372354 4473 leader_election.cc:302] T 00000000000000000000000000000000 P d95894a8c1174b9d80a0f01039daafbb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d95894a8c1174b9d80a0f01039daafbb; no voters:
I20260809 02:02:38.372766 4473 leader_election.cc:288] T 00000000000000000000000000000000 P d95894a8c1174b9d80a0f01039daafbb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:02:38.372869 4479 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d95894a8c1174b9d80a0f01039daafbb [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:02:38.373091 4479 raft_consensus.cc:686] T 00000000000000000000000000000000 P d95894a8c1174b9d80a0f01039daafbb [term 1 LEADER]: Becoming Leader. State: Replica: d95894a8c1174b9d80a0f01039daafbb, State: Running, Role: LEADER
I20260809 02:02:38.373523 4479 consensus_queue.cc:237] T 00000000000000000000000000000000 P d95894a8c1174b9d80a0f01039daafbb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d95894a8c1174b9d80a0f01039daafbb" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 38497 } }
I20260809 02:02:38.373980 4476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:38.359601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36540 (local address 127.1.192.254:38497)
0809 02:02:38.360091 (+ 490us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:38.360107 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:38.360149 (+ 42us) server_negotiation.cc:407] Connection header received
0809 02:02:38.360824 (+ 675us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:38.360842 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:38.361222 (+ 380us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:38.361571 (+ 349us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:38.362498 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:38.363512 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:38.364682 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:38.365219 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:38.365342 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:38.368319 (+ 2977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:02:38.368325 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:02:38.368340 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:02:38.368370 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:02:38.370396 (+ 2026us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:38.370925 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:38.370932 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:38.370938 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:38.371122 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:38.371438 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:38.371442 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:38.371444 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:38.371899 (+ 455us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:02:38.372112 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:38.372438 (+ 326us) server_negotiation.cc:299] Negotiation successful
0809 02:02:38.372808 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":324,"thread_start_us":152,"threads_started":1}
I20260809 02:02:38.374228 4473 sys_catalog.cc:564] T 00000000000000000000000000000000 P d95894a8c1174b9d80a0f01039daafbb [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:02:38.374838 4479 sys_catalog.cc:455] T 00000000000000000000000000000000 P d95894a8c1174b9d80a0f01039daafbb [sys.catalog]: SysCatalogTable state changed. Reason: New leader d95894a8c1174b9d80a0f01039daafbb. Latest consensus state: current_term: 1 leader_uuid: "d95894a8c1174b9d80a0f01039daafbb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d95894a8c1174b9d80a0f01039daafbb" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 38497 } } }
I20260809 02:02:38.375056 4479 sys_catalog.cc:458] T 00000000000000000000000000000000 P d95894a8c1174b9d80a0f01039daafbb [sys.catalog]: This master's current role is: LEADER
I20260809 02:02:38.375311 4480 sys_catalog.cc:455] T 00000000000000000000000000000000 P d95894a8c1174b9d80a0f01039daafbb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d95894a8c1174b9d80a0f01039daafbb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d95894a8c1174b9d80a0f01039daafbb" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 38497 } } }
I20260809 02:02:38.375429 4480 sys_catalog.cc:458] T 00000000000000000000000000000000 P d95894a8c1174b9d80a0f01039daafbb [sys.catalog]: This master's current role is: LEADER
I20260809 02:02:38.375572 4488 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:02:38.378765 4488 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:02:38.384732 4488 catalog_manager.cc:1347] Generated new cluster ID: 4b02496f0bb04b33afbb824e2ff97f99
I20260809 02:02:38.384819 4488 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:02:38.399770 4488 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:02:38.400794 4488 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:02:38.405980 4488 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d95894a8c1174b9d80a0f01039daafbb: Generated new TSK 0
I20260809 02:02:38.406677 4488 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.192.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.192.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.192.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.193.keytab.
Entry for principal kudu/127.1.192.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.193.keytab.
I20260809 02:02:38.441219 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786240941968412-1795-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786240941968412-1795-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786240941968412-1795-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786240941968412-1795-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.192.193:0
--local_ip_for_outbound_sockets=127.1.192.193
--webserver_interface=127.1.192.193
--webserver_port=0
--tserver_master_addrs=127.1.192.254:38497
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.193.keytab
--principal=kudu/127.1.192.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:46231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:02:38.557294 4503 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:02:38.557672 4503 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:02:38.557752 4503 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:02:38.561864 4503 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:02:38.561987 4503 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:02:38.562090 4503 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.192.193
I20260809 02:02:38.567571 4503 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:46231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786240941968412-1795-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786240941968412-1795-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.193.keytab
--principal=kudu/127.1.192.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.192.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786240941968412-1795-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.192.193
--webserver_port=0
--tserver_master_addrs=127.1.192.254:38497
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4503
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.192.193
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786240941968412-1795-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:02:38.568773 4503 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:02:38.569845 4503 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:02:38.580133 4509 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:02:38.580169 4508 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:02:38.580138 4511 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:02:38.580370 4503 server_base.cc:1034] running on GCE node
I20260809 02:02:38.580891 4503 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:02:38.581664 4503 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:02:38.582854 4503 hybrid_clock.cc:648] HybridClock initialized: now 1786240958582821 us; error 45 us; skew 500 ppm
Aug 09 02:02:38 dist-test-slave-ss4n krb5kdc[4379](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240958, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:02:38.585748 4503 init.cc:377] Logged in from keytab as kudu/127.1.192.193@KRBTEST.COM (short username kudu)
I20260809 02:02:38.586644 4503 webserver.cc:460] Webserver started at http://127.1.192.193:44117/ using document root <none> and password file <none>
I20260809 02:02:38.587229 4503 fs_manager.cc:359] Metadata directory not provided
I20260809 02:02:38.587321 4503 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:02:38.587558 4503 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:02:38.589752 4503 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786240941968412-1795-0/minicluster-data/ts-0/data/instance:
uuid: "06ad2a7972bc44898ca6ff42ce9eb505"
format_stamp: "Formatted at 2026-08-09 02:02:38 on dist-test-slave-ss4n"
server_key: "ac2b9b49c873e68a0a4d362bc39412b0"
server_key_iv: "5e761e19f4eae8d6f89cd5561407bbfc"
server_key_version: "encryptionkey@0"
I20260809 02:02:38.590345 4503 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786240941968412-1795-0/minicluster-data/ts-0/wal/instance:
uuid: "06ad2a7972bc44898ca6ff42ce9eb505"
format_stamp: "Formatted at 2026-08-09 02:02:38 on dist-test-slave-ss4n"
server_key: "ac2b9b49c873e68a0a4d362bc39412b0"
server_key_iv: "5e761e19f4eae8d6f89cd5561407bbfc"
server_key_version: "encryptionkey@0"
I20260809 02:02:38.594066 4503 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 02:02:38.596441 4518 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:38.597677 4503 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 02:02:38.597810 4503 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786240941968412-1795-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786240941968412-1795-0/minicluster-data/ts-0/wal
uuid: "06ad2a7972bc44898ca6ff42ce9eb505"
format_stamp: "Formatted at 2026-08-09 02:02:38 on dist-test-slave-ss4n"
server_key: "ac2b9b49c873e68a0a4d362bc39412b0"
server_key_iv: "5e761e19f4eae8d6f89cd5561407bbfc"
server_key_version: "encryptionkey@0"
I20260809 02:02:38.597929 4503 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786240941968412-1795-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786240941968412-1795-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786240941968412-1795-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:02:38.607962 4503 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:02:38.611119 4503 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:02:38.611366 4503 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:02:38.611999 4503 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:02:38.612859 4503 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:02:38.612962 4503 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:38.613055 4503 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:02:38.613109 4503 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:38.622406 4503 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.193:43287
I20260809 02:02:38.622429 4643 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.193:43287 every 8 connection(s)
I20260809 02:02:38.623328 4503 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786240941968412-1795-0/minicluster-data/ts-0/data/info.pb
I20260809 02:02:38.629142 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 4503
I20260809 02:02:38.629321 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786240941968412-1795-0/minicluster-data/ts-0/wal/instance
I20260809 02:02:38.629683 1795 external_mini_cluster.cc:1428] Setting key 8601b163e259cca020671c01e9be389a
Aug 09 02:02:38 dist-test-slave-ss4n krb5kdc[4379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240958, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.193@KRBTEST.COM for kudu/127.1.192.254@KRBTEST.COM
I20260809 02:02:38.638957 4476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:38.625682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.193:60075 (local address 127.1.192.254:38497)
0809 02:02:38.625899 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:38.625903 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:38.626522 (+ 619us) server_negotiation.cc:407] Connection header received
0809 02:02:38.627841 (+ 1319us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:38.627847 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:38.627914 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:38.627989 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:38.629454 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:38.630202 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:38.631562 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:38.631823 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:38.631940 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:38.634979 (+ 3039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:02:38.634982 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:02:38.634984 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:02:38.635009 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:02:38.636504 (+ 1495us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:38.637190 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:38.637192 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:38.637193 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:38.637269 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:38.637731 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:38.637734 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:38.637735 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:38.637918 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:02:38.638020 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:38.638716 (+ 696us) server_negotiation.cc:299] Negotiation successful
0809 02:02:38.638829 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112}
I20260809 02:02:38.640267 4646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:38.625886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:38497 (local address 127.1.192.193:60075)
0809 02:02:38.626364 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:38.626409 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 02:02:38.627609 (+ 1200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:02:38.628151 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:02:38.628158 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:02:38.628515 (+ 357us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:02:38.629211 (+ 696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:38.629234 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:38.630345 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:38.630348 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:38.631423 (+ 1075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:38.631434 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:38.631998 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:38.632002 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:38.632185 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:38.632863 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:02:38.632880 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:02:38.634777 (+ 1897us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:02:38.636630 (+ 1853us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:38.636640 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:38.636661 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:38.637072 (+ 411us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:38.637371 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:38.637376 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:38.637379 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:38.637628 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:38.638035 (+ 407us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:02:38.638045 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:02:38.638417 (+ 372us) client_negotiation.cc:769] Sending connection context
0809 02:02:38.638650 (+ 233us) client_negotiation.cc:240] Negotiation successful
0809 02:02:38.639049 (+ 399us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":121,"threads_started":1}
WARNING: no policy specified for kudu/127.1.192.194@KRBTEST.COM; defaulting to no policy
I20260809 02:02:38.641853 4644 heartbeater.cc:347] Connected to a master server at 127.1.192.254:38497
I20260809 02:02:38.642136 4644 heartbeater.cc:464] Registering TS with master...
I20260809 02:02:38.642689 4644 heartbeater.cc:511] Master 127.1.192.254:38497 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.192.194@KRBTEST.COM" created.
I20260809 02:02:38.644320 4435 ts_manager.cc:194] Registered new tserver with Master: 06ad2a7972bc44898ca6ff42ce9eb505 (127.1.192.193:43287)
I20260809 02:02:38.645998 4435 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.192.193@KRBTEST.COM'} at 127.1.192.193:60075
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.192.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.194.keytab.
Entry for principal kudu/127.1.192.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.194.keytab.
I20260809 02:02:38.665864 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786240941968412-1795-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786240941968412-1795-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786240941968412-1795-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786240941968412-1795-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.192.194:0
--local_ip_for_outbound_sockets=127.1.192.194
--webserver_interface=127.1.192.194
--webserver_port=0
--tserver_master_addrs=127.1.192.254:38497
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.194.keytab
--principal=kudu/127.1.192.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:46231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:02:38.784788 4649 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:02:38.785101 4649 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:02:38.785183 4649 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:02:38.789247 4649 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:02:38.789348 4649 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:02:38.789445 4649 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.192.194
I20260809 02:02:38.794855 4649 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:46231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786240941968412-1795-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786240941968412-1795-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.194.keytab
--principal=kudu/127.1.192.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.192.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786240941968412-1795-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.192.194
--webserver_port=0
--tserver_master_addrs=127.1.192.254:38497
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4649
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.192.194
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786240941968412-1795-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:02:38.796015 4649 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:02:38.797041 4649 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:02:38.806918 4657 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:02:38.806918 4655 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:02:38.806912 4654 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:02:38.807349 4649 server_base.cc:1034] running on GCE node
I20260809 02:02:38.807768 4649 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:02:38.808497 4649 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:02:38.809685 4649 hybrid_clock.cc:648] HybridClock initialized: now 1786240958809669 us; error 30 us; skew 500 ppm
Aug 09 02:02:38 dist-test-slave-ss4n krb5kdc[4379](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240958, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:02:38.812604 4649 init.cc:377] Logged in from keytab as kudu/127.1.192.194@KRBTEST.COM (short username kudu)
I20260809 02:02:38.813572 4649 webserver.cc:460] Webserver started at http://127.1.192.194:45273/ using document root <none> and password file <none>
I20260809 02:02:38.814109 4649 fs_manager.cc:359] Metadata directory not provided
I20260809 02:02:38.814172 4649 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:02:38.814337 4649 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:02:38.816457 4649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786240941968412-1795-0/minicluster-data/ts-1/data/instance:
uuid: "361266a470b248a7a7935ca80d5d16f5"
format_stamp: "Formatted at 2026-08-09 02:02:38 on dist-test-slave-ss4n"
server_key: "8b1b9adb866cf815a4a7e0e3d63529dd"
server_key_iv: "c926a03e84931534744d06f0e09ae5a9"
server_key_version: "encryptionkey@0"
I20260809 02:02:38.816989 4649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786240941968412-1795-0/minicluster-data/ts-1/wal/instance:
uuid: "361266a470b248a7a7935ca80d5d16f5"
format_stamp: "Formatted at 2026-08-09 02:02:38 on dist-test-slave-ss4n"
server_key: "8b1b9adb866cf815a4a7e0e3d63529dd"
server_key_iv: "c926a03e84931534744d06f0e09ae5a9"
server_key_version: "encryptionkey@0"
I20260809 02:02:38.821163 4649 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260809 02:02:38.823757 4665 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:38.824994 4649 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 02:02:38.825095 4649 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786240941968412-1795-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786240941968412-1795-0/minicluster-data/ts-1/wal
uuid: "361266a470b248a7a7935ca80d5d16f5"
format_stamp: "Formatted at 2026-08-09 02:02:38 on dist-test-slave-ss4n"
server_key: "8b1b9adb866cf815a4a7e0e3d63529dd"
server_key_iv: "c926a03e84931534744d06f0e09ae5a9"
server_key_version: "encryptionkey@0"
I20260809 02:02:38.825186 4649 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786240941968412-1795-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786240941968412-1795-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786240941968412-1795-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:02:38.846356 4649 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:02:38.849918 4649 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:02:38.850118 4649 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:02:38.850776 4649 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:02:38.851744 4649 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:02:38.851804 4649 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:38.851853 4649 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:02:38.851887 4649 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:38.861548 4649 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.194:46575
I20260809 02:02:38.861568 4798 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.194:46575 every 8 connection(s)
I20260809 02:02:38.862517 4649 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786240941968412-1795-0/minicluster-data/ts-1/data/info.pb
I20260809 02:02:38.863615 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 4649
I20260809 02:02:38.863703 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786240941968412-1795-0/minicluster-data/ts-1/wal/instance
I20260809 02:02:38.863989 1795 external_mini_cluster.cc:1428] Setting key a131b0f1ac46d23f8e8dcac9fc1f03f7
Aug 09 02:02:38 dist-test-slave-ss4n krb5kdc[4379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240958, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.194@KRBTEST.COM for kudu/127.1.192.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.192.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.192.195@KRBTEST.COM" created.
I20260809 02:02:38.877688 4476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:38.864511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.194:37675 (local address 127.1.192.254:38497)
0809 02:02:38.864864 (+ 353us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:38.864869 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:38.865811 (+ 942us) server_negotiation.cc:407] Connection header received
0809 02:02:38.867170 (+ 1359us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:38.867173 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:38.867231 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:38.867305 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:38.868494 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:38.869054 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:38.870344 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:38.870585 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:38.870702 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:38.873428 (+ 2726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:02:38.873434 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:02:38.873440 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:02:38.873505 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:02:38.875146 (+ 1641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:38.875902 (+ 756us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:38.875905 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:38.875906 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:38.875989 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:38.876329 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:38.876332 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:38.876350 (+ 18us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:38.876635 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:02:38.876744 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:38.877258 (+ 514us) server_negotiation.cc:299] Negotiation successful
0809 02:02:38.877401 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":258}
I20260809 02:02:38.878798 4801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:38.865015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:38497 (local address 127.1.192.194:37675)
0809 02:02:38.865664 (+ 649us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:38.865710 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 02:02:38.866950 (+ 1240us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:02:38.867473 (+ 523us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:02:38.867480 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:02:38.867853 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:02:38.868345 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:38.868358 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:38.869222 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:38.869225 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:38.870207 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:38.870217 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:38.870747 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:38.870751 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:38.870922 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:38.871571 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:02:38.871587 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:02:38.873219 (+ 1632us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:02:38.875312 (+ 2093us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:38.875317 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:38.875331 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:38.875803 (+ 472us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:38.876085 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:38.876088 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:38.876089 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:38.876248 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:38.876752 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:02:38.876757 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:02:38.877003 (+ 246us) client_negotiation.cc:769] Sending connection context
0809 02:02:38.877196 (+ 193us) client_negotiation.cc:240] Negotiation successful
0809 02:02:38.877630 (+ 434us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":158,"threads_started":1}
I20260809 02:02:38.880924 4799 heartbeater.cc:347] Connected to a master server at 127.1.192.254:38497
I20260809 02:02:38.881249 4799 heartbeater.cc:464] Registering TS with master...
I20260809 02:02:38.881943 4799 heartbeater.cc:511] Master 127.1.192.254:38497 requested a full tablet report, sending...
I20260809 02:02:38.883281 4435 ts_manager.cc:194] Registered new tserver with Master: 361266a470b248a7a7935ca80d5d16f5 (127.1.192.194:46575)
I20260809 02:02:38.883934 4435 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.192.194@KRBTEST.COM'} at 127.1.192.194:37675
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.192.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.195.keytab.
Entry for principal kudu/127.1.192.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.195.keytab.
I20260809 02:02:38.897066 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786240941968412-1795-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786240941968412-1795-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786240941968412-1795-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786240941968412-1795-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.192.195:0
--local_ip_for_outbound_sockets=127.1.192.195
--webserver_interface=127.1.192.195
--webserver_port=0
--tserver_master_addrs=127.1.192.254:38497
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.195.keytab
--principal=kudu/127.1.192.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:46231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:02:39.023590 4804 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:02:39.023960 4804 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:02:39.024044 4804 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:02:39.028249 4804 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:02:39.028348 4804 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:02:39.028445 4804 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.192.195
I20260809 02:02:39.034192 4804 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:46231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786240941968412-1795-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786240941968412-1795-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.195.keytab
--principal=kudu/127.1.192.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.192.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786240941968412-1795-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.192.195
--webserver_port=0
--tserver_master_addrs=127.1.192.254:38497
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4804
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.192.195
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786240941968412-1795-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:02:39.035369 4804 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:02:39.036458 4804 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:02:39.046434 4809 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:02:39.046463 4810 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:02:39.046504 4812 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:02:39.046567 4804 server_base.cc:1034] running on GCE node
I20260809 02:02:39.047169 4804 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:02:39.047776 4804 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:02:39.048974 4804 hybrid_clock.cc:648] HybridClock initialized: now 1786240959048963 us; error 52 us; skew 500 ppm
Aug 09 02:02:39 dist-test-slave-ss4n krb5kdc[4379](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240959, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:02:39.052023 4804 init.cc:377] Logged in from keytab as kudu/127.1.192.195@KRBTEST.COM (short username kudu)
I20260809 02:02:39.053017 4804 webserver.cc:460] Webserver started at http://127.1.192.195:43803/ using document root <none> and password file <none>
I20260809 02:02:39.053699 4804 fs_manager.cc:359] Metadata directory not provided
I20260809 02:02:39.053798 4804 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:02:39.054040 4804 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:02:39.056145 4804 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786240941968412-1795-0/minicluster-data/ts-2/data/instance:
uuid: "4349855ccb314f2eaaa88bebfc2db4b4"
format_stamp: "Formatted at 2026-08-09 02:02:39 on dist-test-slave-ss4n"
server_key: "7e9de5b9df521b82662200cc17a5a27c"
server_key_iv: "5f4783b0e312574e488ae3b9256dd93b"
server_key_version: "encryptionkey@0"
I20260809 02:02:39.056752 4804 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786240941968412-1795-0/minicluster-data/ts-2/wal/instance:
uuid: "4349855ccb314f2eaaa88bebfc2db4b4"
format_stamp: "Formatted at 2026-08-09 02:02:39 on dist-test-slave-ss4n"
server_key: "7e9de5b9df521b82662200cc17a5a27c"
server_key_iv: "5f4783b0e312574e488ae3b9256dd93b"
server_key_version: "encryptionkey@0"
I20260809 02:02:39.060551 4804 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260809 02:02:39.063109 4819 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:39.064312 4804 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 02:02:39.064455 4804 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786240941968412-1795-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786240941968412-1795-0/minicluster-data/ts-2/wal
uuid: "4349855ccb314f2eaaa88bebfc2db4b4"
format_stamp: "Formatted at 2026-08-09 02:02:39 on dist-test-slave-ss4n"
server_key: "7e9de5b9df521b82662200cc17a5a27c"
server_key_iv: "5f4783b0e312574e488ae3b9256dd93b"
server_key_version: "encryptionkey@0"
I20260809 02:02:39.064576 4804 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786240941968412-1795-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786240941968412-1795-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786240941968412-1795-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:02:39.089249 4804 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:02:39.093389 4804 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:02:39.093686 4804 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:02:39.094437 4804 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:02:39.095350 4804 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:02:39.095429 4804 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:39.095517 4804 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:02:39.095579 4804 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:39.106621 4804 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.195:33307
I20260809 02:02:39.106642 4943 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.195:33307 every 8 connection(s)
I20260809 02:02:39.107626 4804 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786240941968412-1795-0/minicluster-data/ts-2/data/info.pb
I20260809 02:02:39.115090 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 4804
I20260809 02:02:39.115221 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786240941968412-1795-0/minicluster-data/ts-2/wal/instance
I20260809 02:02:39.115541 1795 external_mini_cluster.cc:1428] Setting key 54b7cf93f57831a84c082ae63d8f8856
Aug 09 02:02:39 dist-test-slave-ss4n krb5kdc[4379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240959, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.195@KRBTEST.COM for kudu/127.1.192.254@KRBTEST.COM
I20260809 02:02:39.122452 4476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:39.109472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.195:45597 (local address 127.1.192.254:38497)
0809 02:02:39.109666 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:39.109672 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:39.110741 (+ 1069us) server_negotiation.cc:407] Connection header received
0809 02:02:39.112067 (+ 1326us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:39.112070 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:39.112121 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:39.112203 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:39.113369 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:39.114138 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:39.115659 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:39.115906 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:39.116001 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:39.119127 (+ 3126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:02:39.119130 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:02:39.119132 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:02:39.119164 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:02:39.120511 (+ 1347us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:39.121007 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:39.121009 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:39.121010 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:39.121089 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:39.121399 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:39.121401 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:39.121403 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:39.121644 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:02:39.121724 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:39.122207 (+ 483us) server_negotiation.cc:299] Negotiation successful
0809 02:02:39.122320 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 02:02:39.123282 4946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:39.110097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:38497 (local address 127.1.192.195:45597)
0809 02:02:39.110567 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:39.110611 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 02:02:39.111833 (+ 1222us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:02:39.112359 (+ 526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:02:39.112365 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:02:39.112745 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:02:39.113214 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:39.113227 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:39.114277 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:39.114280 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:39.115519 (+ 1239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:39.115531 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:39.116080 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:39.116085 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:39.116336 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:39.117088 (+ 752us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:02:39.117109 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:02:39.118968 (+ 1859us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:02:39.120622 (+ 1654us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:39.120627 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:39.120647 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:39.120917 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:39.121173 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:39.121175 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:39.121176 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:39.121319 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:39.121729 (+ 410us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:02:39.121734 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:02:39.121977 (+ 243us) client_negotiation.cc:769] Sending connection context
0809 02:02:39.122140 (+ 163us) client_negotiation.cc:240] Negotiation successful
0809 02:02:39.122427 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":131,"threads_started":1}
I20260809 02:02:39.124418 4944 heartbeater.cc:347] Connected to a master server at 127.1.192.254:38497
I20260809 02:02:39.124693 4944 heartbeater.cc:464] Registering TS with master...
I20260809 02:02:39.125196 4944 heartbeater.cc:511] Master 127.1.192.254:38497 requested a full tablet report, sending...
I20260809 02:02:39.126325 4435 ts_manager.cc:194] Registered new tserver with Master: 4349855ccb314f2eaaa88bebfc2db4b4 (127.1.192.195:33307)
I20260809 02:02:39.126912 4435 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.192.195@KRBTEST.COM'} at 127.1.192.195:45597
I20260809 02:02:39.130084 1795 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 02:02:39.145757 4476 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:02:39.144191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36556 (local address 127.1.192.254:38497)
0809 02:02:39.144358 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:39.144363 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:39.144529 (+ 166us) server_negotiation.cc:407] Connection header received
0809 02:02:39.144632 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:39.144635 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:39.144685 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:39.144760 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:39.145596 (+ 836us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36556: BlockingRecv error: recv got EOF from 127.0.0.1:36556 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 02:02:39.146838 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 4503
I20260809 02:02:39.155853 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 4649
I20260809 02:02:39.162839 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 4804
I20260809 02:02:39.169055 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 4393
2026-08-09T02:02:39Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2266 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786240941968412-1795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:02:39 dist-test-slave-ss4n krb5kdc[4959](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:02:39 dist-test-slave-ss4n krb5kdc[4959](info): set up 2 sockets
Aug 09 02:02:39 dist-test-slave-ss4n krb5kdc[4959](info): commencing operation
krb5kdc: starting...
W20260809 02:02:40.273298 1795 mini_kdc.cc:121] Time spent starting KDC: real 1.079s user 0.007s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786240941968412-1795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786240941968412-1795-0/krb5kdc/test-user.keytab.
Aug 09 02:02:40 dist-test-slave-ss4n krb5kdc[4959](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240960, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:02:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:02:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.192.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.192.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.192.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab.
Entry for principal kudu/127.1.192.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab.
I20260809 02:02:40.410133 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786240941968412-1795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786240941968412-1795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786240941968412-1795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.192.254:38531
--webserver_interface=127.1.192.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab
--principal=kudu/127.1.192.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:43515
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.192.254:38531
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:02:40.528434 4974 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:02:40.528705 4974 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:02:40.528748 4974 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:02:40.532794 4974 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:02:40.532859 4974 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:02:40.532879 4974 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:02:40.532895 4974 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:02:40.532944 4974 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:02:40.538355 4974 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:43515
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786240941968412-1795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786240941968412-1795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.192.254:38531
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab
--principal=kudu/127.1.192.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.192.254:38531
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.192.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4974
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786240941968412-1795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:02:40.539430 4974 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:02:40.540464 4974 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:02:40.549913 4979 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:02:40.549918 4982 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:02:40.550004 4980 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:02:40.550220 4974 server_base.cc:1034] running on GCE node
I20260809 02:02:40.550798 4974 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:02:40.551968 4974 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:02:40.553192 4974 hybrid_clock.cc:648] HybridClock initialized: now 1786240960553158 us; error 45 us; skew 500 ppm
Aug 09 02:02:40 dist-test-slave-ss4n krb5kdc[4959](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240960, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:02:40.560068 4974 init.cc:377] Logged in from keytab as kudu/127.1.192.254@KRBTEST.COM (short username kudu)
I20260809 02:02:40.561088 4974 webserver.cc:460] Webserver started at http://127.1.192.254:40055/ using document root <none> and password file <none>
I20260809 02:02:40.561712 4974 fs_manager.cc:359] Metadata directory not provided
I20260809 02:02:40.561784 4974 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:02:40.561962 4974 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:02:40.564088 4974 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786240941968412-1795-0/minicluster-data/master-0/data/instance:
uuid: "ac28931474554b39b56f3e383145bd35"
format_stamp: "Formatted at 2026-08-09 02:02:40 on dist-test-slave-ss4n"
server_key: "d399c14b40546a83a03d799ce9b91bf2"
server_key_iv: "704e12ed9807a12b631f89553881ad89"
server_key_version: "encryptionkey@0"
I20260809 02:02:40.564620 4974 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786240941968412-1795-0/minicluster-data/master-0/wal/instance:
uuid: "ac28931474554b39b56f3e383145bd35"
format_stamp: "Formatted at 2026-08-09 02:02:40 on dist-test-slave-ss4n"
server_key: "d399c14b40546a83a03d799ce9b91bf2"
server_key_iv: "704e12ed9807a12b631f89553881ad89"
server_key_version: "encryptionkey@0"
I20260809 02:02:40.568616 4974 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 02:02:40.571156 4992 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:40.572495 4974 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260809 02:02:40.572727 4974 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786240941968412-1795-0/minicluster-data/master-0/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786240941968412-1795-0/minicluster-data/master-0/wal
uuid: "ac28931474554b39b56f3e383145bd35"
format_stamp: "Formatted at 2026-08-09 02:02:40 on dist-test-slave-ss4n"
server_key: "d399c14b40546a83a03d799ce9b91bf2"
server_key_iv: "704e12ed9807a12b631f89553881ad89"
server_key_version: "encryptionkey@0"
I20260809 02:02:40.572862 4974 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786240941968412-1795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786240941968412-1795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786240941968412-1795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:02:40.621977 4974 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:02:40.625909 4974 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:02:40.626151 4974 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:02:40.634379 4974 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.254:38531
I20260809 02:02:40.634707 5050 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.254:38531 every 8 connection(s)
I20260809 02:02:40.635501 4974 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
I20260809 02:02:40.638440 5051 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:02:40.638862 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 4974
I20260809 02:02:40.639006 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786240941968412-1795-0/minicluster-data/master-0/wal/instance
I20260809 02:02:40.639307 1795 external_mini_cluster.cc:1428] Setting key f9b3eb616a7e40a98a1753b6c39331d8
I20260809 02:02:40.645258 5051 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac28931474554b39b56f3e383145bd35: Bootstrap starting.
I20260809 02:02:40.647936 5051 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ac28931474554b39b56f3e383145bd35: Neither blocks nor log segments found. Creating new log.
I20260809 02:02:40.648731 5051 log.cc:824] T 00000000000000000000000000000000 P ac28931474554b39b56f3e383145bd35: Log is configured to *not* fsync() on all Append() calls
Aug 09 02:02:40 dist-test-slave-ss4n krb5kdc[4959](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240960, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.192.254@KRBTEST.COM
I20260809 02:02:40.651304 5051 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac28931474554b39b56f3e383145bd35: No bootstrap required, opened a new log
I20260809 02:02:40.656320 5055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:40.640799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33002 (local address 127.1.192.254:38531)
0809 02:02:40.641272 (+ 473us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:40.641291 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:40.641338 (+ 47us) server_negotiation.cc:407] Connection header received
0809 02:02:40.642225 (+ 887us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:40.642249 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:40.642668 (+ 419us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:40.643038 (+ 370us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:40.643770 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:40.645244 (+ 1474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:40.646418 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:40.647119 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:40.647284 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:40.650529 (+ 3245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:02:40.650536 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:02:40.650552 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:02:40.650584 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:02:40.652755 (+ 2171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:40.653253 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:40.653260 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:40.653268 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:40.653410 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:40.653836 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:40.653841 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:40.653844 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:40.654428 (+ 584us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:02:40.654623 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:40.654962 (+ 339us) server_negotiation.cc:299] Negotiation successful
0809 02:02:40.655285 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":313,"thread_start_us":124,"threads_started":1}
I20260809 02:02:40.656292 5051 raft_consensus.cc:348] T 00000000000000000000000000000000 P ac28931474554b39b56f3e383145bd35 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac28931474554b39b56f3e383145bd35" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 38531 } }
I20260809 02:02:40.656574 5051 raft_consensus.cc:374] T 00000000000000000000000000000000 P ac28931474554b39b56f3e383145bd35 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:02:40.656667 5051 raft_consensus.cc:729] T 00000000000000000000000000000000 P ac28931474554b39b56f3e383145bd35 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac28931474554b39b56f3e383145bd35, State: Initialized, Role: FOLLOWER
I20260809 02:02:40.657179 5051 consensus_queue.cc:260] T 00000000000000000000000000000000 P ac28931474554b39b56f3e383145bd35 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac28931474554b39b56f3e383145bd35" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 38531 } }
I20260809 02:02:40.657343 5051 raft_consensus.cc:388] T 00000000000000000000000000000000 P ac28931474554b39b56f3e383145bd35 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:02:40.657433 5051 raft_consensus.cc:482] T 00000000000000000000000000000000 P ac28931474554b39b56f3e383145bd35 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:02:40.657567 5051 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ac28931474554b39b56f3e383145bd35 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:02:40.658602 5051 raft_consensus.cc:504] T 00000000000000000000000000000000 P ac28931474554b39b56f3e383145bd35 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac28931474554b39b56f3e383145bd35" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 38531 } }
I20260809 02:02:40.658970 5051 leader_election.cc:302] T 00000000000000000000000000000000 P ac28931474554b39b56f3e383145bd35 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ac28931474554b39b56f3e383145bd35; no voters:
I20260809 02:02:40.659343 5051 leader_election.cc:288] T 00000000000000000000000000000000 P ac28931474554b39b56f3e383145bd35 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:02:40.659507 5057 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ac28931474554b39b56f3e383145bd35 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:02:40.659806 5057 raft_consensus.cc:686] T 00000000000000000000000000000000 P ac28931474554b39b56f3e383145bd35 [term 1 LEADER]: Becoming Leader. State: Replica: ac28931474554b39b56f3e383145bd35, State: Running, Role: LEADER
I20260809 02:02:40.659924 1795 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 02:02:40.660203 5057 consensus_queue.cc:237] T 00000000000000000000000000000000 P ac28931474554b39b56f3e383145bd35 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac28931474554b39b56f3e383145bd35" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 38531 } }
I20260809 02:02:40.660812 5051 sys_catalog.cc:564] T 00000000000000000000000000000000 P ac28931474554b39b56f3e383145bd35 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:02:40.662107 5058 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac28931474554b39b56f3e383145bd35 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ac28931474554b39b56f3e383145bd35" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac28931474554b39b56f3e383145bd35" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 38531 } } }
I20260809 02:02:40.662089 5060 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac28931474554b39b56f3e383145bd35 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ac28931474554b39b56f3e383145bd35. Latest consensus state: current_term: 1 leader_uuid: "ac28931474554b39b56f3e383145bd35" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac28931474554b39b56f3e383145bd35" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 38531 } } }
I20260809 02:02:40.662268 5058 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac28931474554b39b56f3e383145bd35 [sys.catalog]: This master's current role is: LEADER
I20260809 02:02:40.662268 5060 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac28931474554b39b56f3e383145bd35 [sys.catalog]: This master's current role is: LEADER
W20260809 02:02:40.666759 5077 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ac28931474554b39b56f3e383145bd35: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 02:02:40.666855 5077 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 02:02:40.666952 5069 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:02:40.667925 5069 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:02:40.674573 5069 catalog_manager.cc:1347] Generated new cluster ID: b208ba4f9a8a49fd822e5bb08c58757f
I20260809 02:02:40.674646 5069 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:02:40.682358 5055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:40.676681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33010 (local address 127.1.192.254:38531)
0809 02:02:40.676824 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:40.676832 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:40.677233 (+ 401us) server_negotiation.cc:407] Connection header received
0809 02:02:40.677436 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:40.677441 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:40.677547 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:40.677651 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:40.678307 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:40.678873 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:40.679867 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:40.680150 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:40.680279 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:40.681242 (+ 963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:02:40.681246 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:02:40.681249 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:02:40.681796 (+ 547us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:02:40.681827 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:02:40.681846 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:02:40.681958 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:40.682095 (+ 137us) server_negotiation.cc:299] Negotiation successful
0809 02:02:40.682161 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 02:02:40.713676 5069 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:02:40.715165 5069 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:02:40.727792 5069 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ac28931474554b39b56f3e383145bd35: Generated new TSK 0
I20260809 02:02:40.728652 5069 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:02:40.743295 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 4974
2026-08-09T02:02:40Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1572 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786240941968412-1795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:02:40 dist-test-slave-ss4n krb5kdc[5090](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:02:40 dist-test-slave-ss4n krb5kdc[5090](info): set up 2 sockets
Aug 09 02:02:40 dist-test-slave-ss4n krb5kdc[5090](info): commencing operation
krb5kdc: starting...
W20260809 02:02:41.909261 1795 mini_kdc.cc:121] Time spent starting KDC: real 1.143s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786240941968412-1795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786240941968412-1795-0/krb5kdc/test-user.keytab.
Aug 09 02:02:42 dist-test-slave-ss4n krb5kdc[5090](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240962, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:02:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:02:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.192.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.192.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.192.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab.
Entry for principal kudu/127.1.192.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab.
I20260809 02:02:42.049822 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786240941968412-1795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786240941968412-1795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786240941968412-1795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.192.254:39267
--webserver_interface=127.1.192.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab
--principal=kudu/127.1.192.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:36613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.192.254:39267
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:02:42.163229 5104 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:02:42.163581 5104 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:02:42.163676 5104 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:02:42.168005 5104 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:02:42.168103 5104 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:02:42.168141 5104 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:02:42.168182 5104 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:02:42.168219 5104 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:02:42.173676 5104 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:36613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786240941968412-1795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786240941968412-1795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.192.254:39267
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab
--principal=kudu/127.1.192.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.192.254:39267
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.192.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5104
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786240941968412-1795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:02:42.174829 5104 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:02:42.175940 5104 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:02:42.185510 5112 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:02:42.185806 5104 server_base.cc:1034] running on GCE node
W20260809 02:02:42.185447 5111 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:02:42.185456 5114 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:02:42.186542 5104 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:02:42.187704 5104 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:02:42.188869 5104 hybrid_clock.cc:648] HybridClock initialized: now 1786240962188859 us; error 29 us; skew 500 ppm
Aug 09 02:02:42 dist-test-slave-ss4n krb5kdc[5090](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240962, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:02:42.191952 5104 init.cc:377] Logged in from keytab as kudu/127.1.192.254@KRBTEST.COM (short username kudu)
I20260809 02:02:42.192961 5104 webserver.cc:460] Webserver started at http://127.1.192.254:38009/ using document root <none> and password file <none>
I20260809 02:02:42.193614 5104 fs_manager.cc:359] Metadata directory not provided
I20260809 02:02:42.193686 5104 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:02:42.193915 5104 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:02:42.196007 5104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786240941968412-1795-0/minicluster-data/master-0/data/instance:
uuid: "8dca6edbf23b4963ae5a5bc55a3f5c74"
format_stamp: "Formatted at 2026-08-09 02:02:42 on dist-test-slave-ss4n"
server_key: "1f22588985827d92a95998a235778b5d"
server_key_iv: "e6d46946b9313530b46eb7601385614e"
server_key_version: "encryptionkey@0"
I20260809 02:02:42.196615 5104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786240941968412-1795-0/minicluster-data/master-0/wal/instance:
uuid: "8dca6edbf23b4963ae5a5bc55a3f5c74"
format_stamp: "Formatted at 2026-08-09 02:02:42 on dist-test-slave-ss4n"
server_key: "1f22588985827d92a95998a235778b5d"
server_key_iv: "e6d46946b9313530b46eb7601385614e"
server_key_version: "encryptionkey@0"
I20260809 02:02:42.200466 5104 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260809 02:02:42.203020 5121 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:42.204228 5104 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 02:02:42.204370 5104 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786240941968412-1795-0/minicluster-data/master-0/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786240941968412-1795-0/minicluster-data/master-0/wal
uuid: "8dca6edbf23b4963ae5a5bc55a3f5c74"
format_stamp: "Formatted at 2026-08-09 02:02:42 on dist-test-slave-ss4n"
server_key: "1f22588985827d92a95998a235778b5d"
server_key_iv: "e6d46946b9313530b46eb7601385614e"
server_key_version: "encryptionkey@0"
I20260809 02:02:42.204488 5104 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786240941968412-1795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786240941968412-1795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786240941968412-1795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:02:42.218632 5104 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:02:42.222129 5104 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:02:42.222391 5104 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:02:42.230113 5104 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.254:39267
I20260809 02:02:42.230440 5183 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.254:39267 every 8 connection(s)
I20260809 02:02:42.231154 5104 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
I20260809 02:02:42.234247 5184 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:02:42.237172 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 5104
I20260809 02:02:42.237313 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786240941968412-1795-0/minicluster-data/master-0/wal/instance
I20260809 02:02:42.237695 1795 external_mini_cluster.cc:1428] Setting key 350872a3afa857b88373b2881f5da177
I20260809 02:02:42.240856 5184 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8dca6edbf23b4963ae5a5bc55a3f5c74: Bootstrap starting.
I20260809 02:02:42.243714 5184 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8dca6edbf23b4963ae5a5bc55a3f5c74: Neither blocks nor log segments found. Creating new log.
I20260809 02:02:42.244626 5184 log.cc:824] T 00000000000000000000000000000000 P 8dca6edbf23b4963ae5a5bc55a3f5c74: Log is configured to *not* fsync() on all Append() calls
I20260809 02:02:42.246665 5184 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8dca6edbf23b4963ae5a5bc55a3f5c74: No bootstrap required, opened a new log
Aug 09 02:02:42 dist-test-slave-ss4n krb5kdc[5090](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240962, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.192.254@KRBTEST.COM
I20260809 02:02:42.251057 5184 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8dca6edbf23b4963ae5a5bc55a3f5c74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8dca6edbf23b4963ae5a5bc55a3f5c74" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 39267 } }
I20260809 02:02:42.251371 5184 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8dca6edbf23b4963ae5a5bc55a3f5c74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:02:42.251437 5184 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8dca6edbf23b4963ae5a5bc55a3f5c74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8dca6edbf23b4963ae5a5bc55a3f5c74, State: Initialized, Role: FOLLOWER
I20260809 02:02:42.252000 5184 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8dca6edbf23b4963ae5a5bc55a3f5c74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8dca6edbf23b4963ae5a5bc55a3f5c74" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 39267 } }
I20260809 02:02:42.252174 5184 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8dca6edbf23b4963ae5a5bc55a3f5c74 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:02:42.252252 5184 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8dca6edbf23b4963ae5a5bc55a3f5c74 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:02:42.252362 5184 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8dca6edbf23b4963ae5a5bc55a3f5c74 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:02:42.253628 5184 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8dca6edbf23b4963ae5a5bc55a3f5c74 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8dca6edbf23b4963ae5a5bc55a3f5c74" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 39267 } }
I20260809 02:02:42.253984 5184 leader_election.cc:302] T 00000000000000000000000000000000 P 8dca6edbf23b4963ae5a5bc55a3f5c74 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8dca6edbf23b4963ae5a5bc55a3f5c74; no voters:
I20260809 02:02:42.254153 5187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:42.239174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39334 (local address 127.1.192.254:39267)
0809 02:02:42.239592 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:42.239608 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:42.239645 (+ 37us) server_negotiation.cc:407] Connection header received
0809 02:02:42.240282 (+ 637us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:42.240299 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:42.240617 (+ 318us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:42.240887 (+ 270us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:42.241922 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:42.242932 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:42.244211 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:42.244665 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:42.244858 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:42.247742 (+ 2884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:02:42.247749 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:02:42.247765 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:02:42.247796 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:02:42.250758 (+ 2962us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:42.251224 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:42.251230 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:42.251236 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:42.251386 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:42.251721 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:42.251725 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:42.251727 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:42.252258 (+ 531us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:02:42.252478 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:42.252780 (+ 302us) server_negotiation.cc:299] Negotiation successful
0809 02:02:42.253084 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":272,"thread_start_us":139,"threads_started":1}
I20260809 02:02:42.254350 5184 leader_election.cc:288] T 00000000000000000000000000000000 P 8dca6edbf23b4963ae5a5bc55a3f5c74 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:02:42.254531 5189 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8dca6edbf23b4963ae5a5bc55a3f5c74 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:02:42.254798 5189 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8dca6edbf23b4963ae5a5bc55a3f5c74 [term 1 LEADER]: Becoming Leader. State: Replica: 8dca6edbf23b4963ae5a5bc55a3f5c74, State: Running, Role: LEADER
I20260809 02:02:42.255179 5189 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8dca6edbf23b4963ae5a5bc55a3f5c74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8dca6edbf23b4963ae5a5bc55a3f5c74" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 39267 } }
I20260809 02:02:42.255616 5184 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8dca6edbf23b4963ae5a5bc55a3f5c74 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:02:42.257702 5190 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8dca6edbf23b4963ae5a5bc55a3f5c74 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8dca6edbf23b4963ae5a5bc55a3f5c74" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8dca6edbf23b4963ae5a5bc55a3f5c74" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 39267 } } }
I20260809 02:02:42.257854 5190 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8dca6edbf23b4963ae5a5bc55a3f5c74 [sys.catalog]: This master's current role is: LEADER
I20260809 02:02:42.258131 5191 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8dca6edbf23b4963ae5a5bc55a3f5c74 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8dca6edbf23b4963ae5a5bc55a3f5c74. Latest consensus state: current_term: 1 leader_uuid: "8dca6edbf23b4963ae5a5bc55a3f5c74" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8dca6edbf23b4963ae5a5bc55a3f5c74" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 39267 } } }
I20260809 02:02:42.258500 5191 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8dca6edbf23b4963ae5a5bc55a3f5c74 [sys.catalog]: This master's current role is: LEADER
W20260809 02:02:42.260439 5206 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8dca6edbf23b4963ae5a5bc55a3f5c74: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 02:02:42.260545 5206 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 02:02:42.260703 5208 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:02:42.261729 5208 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:02:42.268133 5208 catalog_manager.cc:1347] Generated new cluster ID: 6ec9bc9b30a14ce5bd70352ea71a7c7b
I20260809 02:02:42.268221 5208 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:02:42.277161 5208 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:02:42.278157 5208 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:02:42.289450 5208 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8dca6edbf23b4963ae5a5bc55a3f5c74: Generated new TSK 0
I20260809 02:02:42.290261 5208 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.192.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.192.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.192.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.193.keytab.
Entry for principal kudu/127.1.192.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.193.keytab.
I20260809 02:02:42.329885 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786240941968412-1795-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786240941968412-1795-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786240941968412-1795-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786240941968412-1795-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.192.193:0
--local_ip_for_outbound_sockets=127.1.192.193
--webserver_interface=127.1.192.193
--webserver_port=0
--tserver_master_addrs=127.1.192.254:39267
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.193.keytab
--principal=kudu/127.1.192.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:36613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:02:42.445070 5213 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:02:42.445413 5213 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:02:42.445466 5213 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:02:42.449613 5213 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:02:42.449681 5213 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:02:42.449741 5213 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.192.193
I20260809 02:02:42.455145 5213 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:36613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786240941968412-1795-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786240941968412-1795-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.193.keytab
--principal=kudu/127.1.192.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.192.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786240941968412-1795-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.192.193
--webserver_port=0
--tserver_master_addrs=127.1.192.254:39267
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5213
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.192.193
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786240941968412-1795-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:02:42.456280 5213 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:02:42.457369 5213 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:02:42.467295 5221 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:02:42.467284 5218 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:02:42.467387 5213 server_base.cc:1034] running on GCE node
W20260809 02:02:42.467284 5219 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:02:42.467996 5213 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:02:42.468740 5213 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:02:42.469934 5213 hybrid_clock.cc:648] HybridClock initialized: now 1786240962469906 us; error 43 us; skew 500 ppm
Aug 09 02:02:42 dist-test-slave-ss4n krb5kdc[5090](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240962, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:02:42.472811 5213 init.cc:377] Logged in from keytab as kudu/127.1.192.193@KRBTEST.COM (short username kudu)
I20260809 02:02:42.474026 5213 webserver.cc:460] Webserver started at http://127.1.192.193:42115/ using document root <none> and password file <none>
I20260809 02:02:42.474679 5213 fs_manager.cc:359] Metadata directory not provided
I20260809 02:02:42.474792 5213 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:02:42.475075 5213 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:02:42.477754 5213 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786240941968412-1795-0/minicluster-data/ts-0/data/instance:
uuid: "d08f7f4ec4f441c88cc4afbd2dd9b9de"
format_stamp: "Formatted at 2026-08-09 02:02:42 on dist-test-slave-ss4n"
server_key: "5f0ec5ad236b6db2c2e2aaae96e444bd"
server_key_iv: "67cdf9669cc313c5ab82956bdf3a9aff"
server_key_version: "encryptionkey@0"
I20260809 02:02:42.478411 5213 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786240941968412-1795-0/minicluster-data/ts-0/wal/instance:
uuid: "d08f7f4ec4f441c88cc4afbd2dd9b9de"
format_stamp: "Formatted at 2026-08-09 02:02:42 on dist-test-slave-ss4n"
server_key: "5f0ec5ad236b6db2c2e2aaae96e444bd"
server_key_iv: "67cdf9669cc313c5ab82956bdf3a9aff"
server_key_version: "encryptionkey@0"
I20260809 02:02:42.482635 5213 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 02:02:42.485127 5228 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:42.486409 5213 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 02:02:42.486550 5213 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786240941968412-1795-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786240941968412-1795-0/minicluster-data/ts-0/wal
uuid: "d08f7f4ec4f441c88cc4afbd2dd9b9de"
format_stamp: "Formatted at 2026-08-09 02:02:42 on dist-test-slave-ss4n"
server_key: "5f0ec5ad236b6db2c2e2aaae96e444bd"
server_key_iv: "67cdf9669cc313c5ab82956bdf3a9aff"
server_key_version: "encryptionkey@0"
I20260809 02:02:42.486685 5213 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786240941968412-1795-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786240941968412-1795-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786240941968412-1795-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:02:42.512068 5213 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:02:42.515530 5213 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:02:42.515699 5213 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:02:42.516279 5213 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:02:42.517160 5213 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:02:42.517232 5213 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:42.517277 5213 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:02:42.517304 5213 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:42.526540 5213 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.193:33465
I20260809 02:02:42.526561 5358 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.193:33465 every 8 connection(s)
I20260809 02:02:42.527598 5213 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786240941968412-1795-0/minicluster-data/ts-0/data/info.pb
I20260809 02:02:42.537722 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 5213
I20260809 02:02:42.537832 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786240941968412-1795-0/minicluster-data/ts-0/wal/instance
Aug 09 02:02:42 dist-test-slave-ss4n krb5kdc[5090](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240962, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.193@KRBTEST.COM for kudu/127.1.192.254@KRBTEST.COM
I20260809 02:02:42.538168 1795 external_mini_cluster.cc:1428] Setting key 7524ef8709414798e8c88084bcce6e97
I20260809 02:02:42.542948 5187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:42.529631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.193:36301 (local address 127.1.192.254:39267)
0809 02:02:42.529796 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:42.529802 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:42.530750 (+ 948us) server_negotiation.cc:407] Connection header received
0809 02:02:42.532274 (+ 1524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:42.532276 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:42.532323 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:42.532400 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:42.533805 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:42.534424 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:42.535633 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:42.535877 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:42.535962 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:42.538725 (+ 2763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:02:42.538728 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:02:42.538730 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:02:42.538752 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:02:42.540294 (+ 1542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:42.540969 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:42.540972 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:42.540973 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:42.541052 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:42.541591 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:42.541593 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:42.541594 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:42.541857 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:02:42.541974 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:42.542529 (+ 555us) server_negotiation.cc:299] Negotiation successful
0809 02:02:42.542724 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 02:02:42.543624 5361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:42.530003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:39267 (local address 127.1.192.193:36301)
0809 02:02:42.530581 (+ 578us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:42.530630 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 02:02:42.532072 (+ 1442us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:02:42.532640 (+ 568us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:02:42.532647 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:02:42.533036 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:02:42.533626 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:42.533644 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:42.534625 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:42.534628 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:42.535502 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:42.535512 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:42.536018 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:42.536021 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:42.536158 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:42.536825 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:02:42.536841 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:02:42.538495 (+ 1654us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:02:42.540420 (+ 1925us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:42.540429 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:42.540449 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:42.540858 (+ 409us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:42.541149 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:42.541154 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:42.541159 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:42.541424 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:42.541985 (+ 561us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:02:42.541991 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:02:42.542267 (+ 276us) client_negotiation.cc:769] Sending connection context
0809 02:02:42.542427 (+ 160us) client_negotiation.cc:240] Negotiation successful
0809 02:02:42.542708 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":129,"threads_started":1}
I20260809 02:02:42.545004 5359 heartbeater.cc:347] Connected to a master server at 127.1.192.254:39267
I20260809 02:02:42.545269 5359 heartbeater.cc:464] Registering TS with master...
I20260809 02:02:42.545832 5359 heartbeater.cc:511] Master 127.1.192.254:39267 requested a full tablet report, sending...
I20260809 02:02:42.547379 5141 ts_manager.cc:194] Registered new tserver with Master: d08f7f4ec4f441c88cc4afbd2dd9b9de (127.1.192.193:33465)
I20260809 02:02:42.548499 5141 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.192.193@KRBTEST.COM'} at 127.1.192.193:36301
WARNING: no policy specified for kudu/127.1.192.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.192.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.192.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.194.keytab.
Entry for principal kudu/127.1.192.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.194.keytab.
I20260809 02:02:42.573562 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786240941968412-1795-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786240941968412-1795-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786240941968412-1795-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786240941968412-1795-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.192.194:0
--local_ip_for_outbound_sockets=127.1.192.194
--webserver_interface=127.1.192.194
--webserver_port=0
--tserver_master_addrs=127.1.192.254:39267
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.194.keytab
--principal=kudu/127.1.192.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:36613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:02:42.688063 5365 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:02:42.688385 5365 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:02:42.688484 5365 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:02:42.693313 5365 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:02:42.693423 5365 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:02:42.693584 5365 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.192.194
I20260809 02:02:42.699854 5365 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:36613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786240941968412-1795-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786240941968412-1795-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.194.keytab
--principal=kudu/127.1.192.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.192.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786240941968412-1795-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.192.194
--webserver_port=0
--tserver_master_addrs=127.1.192.254:39267
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5365
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.192.194
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786240941968412-1795-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:02:42.701086 5365 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:02:42.702320 5365 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 02:02:42.713529 5365 server_base.cc:1034] running on GCE node
W20260809 02:02:42.713442 5370 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:02:42.713538 5373 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:02:42.713459 5371 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:02:42.714361 5365 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:02:42.715086 5365 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:02:42.716297 5365 hybrid_clock.cc:648] HybridClock initialized: now 1786240962716276 us; error 33 us; skew 500 ppm
Aug 09 02:02:42 dist-test-slave-ss4n krb5kdc[5090](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240962, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:02:42.719415 5365 init.cc:377] Logged in from keytab as kudu/127.1.192.194@KRBTEST.COM (short username kudu)
I20260809 02:02:42.720420 5365 webserver.cc:460] Webserver started at http://127.1.192.194:37899/ using document root <none> and password file <none>
I20260809 02:02:42.721062 5365 fs_manager.cc:359] Metadata directory not provided
I20260809 02:02:42.721165 5365 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:02:42.721424 5365 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:02:42.723738 5365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786240941968412-1795-0/minicluster-data/ts-1/data/instance:
uuid: "86105db01c3548158c3f8a0438751659"
format_stamp: "Formatted at 2026-08-09 02:02:42 on dist-test-slave-ss4n"
server_key: "3fbfb8035ff6d6def257cf9282e3d8aa"
server_key_iv: "f5e1699e216a3296a7aa66036ee0bc93"
server_key_version: "encryptionkey@0"
I20260809 02:02:42.724359 5365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786240941968412-1795-0/minicluster-data/ts-1/wal/instance:
uuid: "86105db01c3548158c3f8a0438751659"
format_stamp: "Formatted at 2026-08-09 02:02:42 on dist-test-slave-ss4n"
server_key: "3fbfb8035ff6d6def257cf9282e3d8aa"
server_key_iv: "f5e1699e216a3296a7aa66036ee0bc93"
server_key_version: "encryptionkey@0"
I20260809 02:02:42.728543 5365 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260809 02:02:42.731228 5380 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:42.732504 5365 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 02:02:42.732671 5365 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786240941968412-1795-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786240941968412-1795-0/minicluster-data/ts-1/wal
uuid: "86105db01c3548158c3f8a0438751659"
format_stamp: "Formatted at 2026-08-09 02:02:42 on dist-test-slave-ss4n"
server_key: "3fbfb8035ff6d6def257cf9282e3d8aa"
server_key_iv: "f5e1699e216a3296a7aa66036ee0bc93"
server_key_version: "encryptionkey@0"
I20260809 02:02:42.732798 5365 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786240941968412-1795-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786240941968412-1795-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786240941968412-1795-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:02:42.746779 5365 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:02:42.750790 5365 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:02:42.751024 5365 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:02:42.751693 5365 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:02:42.752619 5365 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:02:42.752697 5365 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:42.752776 5365 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:02:42.752836 5365 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:42.763465 5365 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.194:35355
I20260809 02:02:42.763480 5506 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.194:35355 every 8 connection(s)
I20260809 02:02:42.764489 5365 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786240941968412-1795-0/minicluster-data/ts-1/data/info.pb
I20260809 02:02:42.771579 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 5365
I20260809 02:02:42.771719 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786240941968412-1795-0/minicluster-data/ts-1/wal/instance
I20260809 02:02:42.772051 1795 external_mini_cluster.cc:1428] Setting key 1595922975dcfcf4d87de5b8a8c9f280
Aug 09 02:02:42 dist-test-slave-ss4n krb5kdc[5090](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240962, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.194@KRBTEST.COM for kudu/127.1.192.254@KRBTEST.COM
I20260809 02:02:42.781919 5187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:42.766365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.194:34327 (local address 127.1.192.254:39267)
0809 02:02:42.766499 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:42.766503 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:42.767748 (+ 1245us) server_negotiation.cc:407] Connection header received
0809 02:02:42.769594 (+ 1846us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:42.769600 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:42.769687 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:42.769818 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:42.771178 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:42.772154 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:42.773424 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:42.773807 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:42.773900 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:42.777372 (+ 3472us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:02:42.777379 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:02:42.777386 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:02:42.777422 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:02:42.779178 (+ 1756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:42.779909 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:42.779916 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:42.779921 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:42.780048 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:42.780466 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:42.780472 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:42.780477 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:42.780787 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:02:42.780921 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:42.781534 (+ 613us) server_negotiation.cc:299] Negotiation successful
0809 02:02:42.781698 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 02:02:42.782931 5510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:42.766981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:39267 (local address 127.1.192.194:34327)
0809 02:02:42.767591 (+ 610us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:42.767634 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 02:02:42.769321 (+ 1687us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:02:42.770026 (+ 705us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:02:42.770035 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:02:42.770516 (+ 481us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:02:42.771023 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:42.771036 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:42.772281 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:42.772284 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:42.773279 (+ 995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:42.773289 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:42.773975 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:42.773980 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:42.774221 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:42.774943 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:02:42.774969 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:02:42.777178 (+ 2209us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:02:42.779305 (+ 2127us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:42.779316 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:42.779336 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:42.779759 (+ 423us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:42.780185 (+ 426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:42.780188 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:42.780190 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:42.780346 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:42.780925 (+ 579us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:02:42.780931 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:02:42.781207 (+ 276us) client_negotiation.cc:769] Sending connection context
0809 02:02:42.781389 (+ 182us) client_negotiation.cc:240] Negotiation successful
0809 02:02:42.781867 (+ 478us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":179,"threads_started":1}
WARNING: no policy specified for kudu/127.1.192.195@KRBTEST.COM; defaulting to no policy
I20260809 02:02:42.784308 5507 heartbeater.cc:347] Connected to a master server at 127.1.192.254:39267
I20260809 02:02:42.784583 5507 heartbeater.cc:464] Registering TS with master...
I20260809 02:02:42.785077 5507 heartbeater.cc:511] Master 127.1.192.254:39267 requested a full tablet report, sending...
I20260809 02:02:42.786161 5141 ts_manager.cc:194] Registered new tserver with Master: 86105db01c3548158c3f8a0438751659 (127.1.192.194:35355)
I20260809 02:02:42.786784 5141 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.192.194@KRBTEST.COM'} at 127.1.192.194:34327
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.192.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.192.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.195.keytab.
Entry for principal kudu/127.1.192.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.195.keytab.
I20260809 02:02:42.805179 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786240941968412-1795-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786240941968412-1795-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786240941968412-1795-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786240941968412-1795-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.192.195:0
--local_ip_for_outbound_sockets=127.1.192.195
--webserver_interface=127.1.192.195
--webserver_port=0
--tserver_master_addrs=127.1.192.254:39267
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.195.keytab
--principal=kudu/127.1.192.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:36613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:02:42.924661 5513 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:02:42.924947 5513 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:02:42.924993 5513 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:02:42.929076 5513 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:02:42.929147 5513 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:02:42.929207 5513 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.192.195
I20260809 02:02:42.934360 5513 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:36613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786240941968412-1795-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786240941968412-1795-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.195.keytab
--principal=kudu/127.1.192.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.192.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786240941968412-1795-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.192.195
--webserver_port=0
--tserver_master_addrs=127.1.192.254:39267
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5513
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.192.195
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786240941968412-1795-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:02:42.935354 5513 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:02:42.936455 5513 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 02:02:42.946596 5513 server_base.cc:1034] running on GCE node
W20260809 02:02:42.946491 5519 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:02:42.946496 5522 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:02:42.946493 5520 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:02:42.947289 5513 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:02:42.947964 5513 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:02:42.949184 5513 hybrid_clock.cc:648] HybridClock initialized: now 1786240962949169 us; error 30 us; skew 500 ppm
Aug 09 02:02:42 dist-test-slave-ss4n krb5kdc[5090](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240962, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:02:42.952078 5513 init.cc:377] Logged in from keytab as kudu/127.1.192.195@KRBTEST.COM (short username kudu)
I20260809 02:02:42.953016 5513 webserver.cc:460] Webserver started at http://127.1.192.195:34093/ using document root <none> and password file <none>
I20260809 02:02:42.953670 5513 fs_manager.cc:359] Metadata directory not provided
I20260809 02:02:42.953769 5513 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:02:42.954008 5513 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:02:42.956218 5513 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786240941968412-1795-0/minicluster-data/ts-2/data/instance:
uuid: "75c281901357485fae5835be2473f1a6"
format_stamp: "Formatted at 2026-08-09 02:02:42 on dist-test-slave-ss4n"
server_key: "ae1268f8b5f11d3804b878be8b036a3e"
server_key_iv: "9566b81c13608162c5c83c11ffb4c24b"
server_key_version: "encryptionkey@0"
I20260809 02:02:42.956854 5513 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786240941968412-1795-0/minicluster-data/ts-2/wal/instance:
uuid: "75c281901357485fae5835be2473f1a6"
format_stamp: "Formatted at 2026-08-09 02:02:42 on dist-test-slave-ss4n"
server_key: "ae1268f8b5f11d3804b878be8b036a3e"
server_key_iv: "9566b81c13608162c5c83c11ffb4c24b"
server_key_version: "encryptionkey@0"
I20260809 02:02:42.960843 5513 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260809 02:02:42.963690 5530 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:42.964919 5513 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 02:02:42.965065 5513 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786240941968412-1795-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786240941968412-1795-0/minicluster-data/ts-2/wal
uuid: "75c281901357485fae5835be2473f1a6"
format_stamp: "Formatted at 2026-08-09 02:02:42 on dist-test-slave-ss4n"
server_key: "ae1268f8b5f11d3804b878be8b036a3e"
server_key_iv: "9566b81c13608162c5c83c11ffb4c24b"
server_key_version: "encryptionkey@0"
I20260809 02:02:42.965209 5513 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786240941968412-1795-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786240941968412-1795-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786240941968412-1795-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:02:42.975088 5513 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:02:42.978360 5513 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:02:42.978583 5513 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:02:42.979208 5513 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:02:42.980060 5513 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:02:42.980154 5513 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:42.980244 5513 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:02:42.980295 5513 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:42.989729 5513 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.195:42999
I20260809 02:02:42.989759 5656 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.195:42999 every 8 connection(s)
I20260809 02:02:42.990746 5513 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786240941968412-1795-0/minicluster-data/ts-2/data/info.pb
I20260809 02:02:42.993028 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 5513
I20260809 02:02:42.993160 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786240941968412-1795-0/minicluster-data/ts-2/wal/instance
I20260809 02:02:42.993521 1795 external_mini_cluster.cc:1428] Setting key 843842d29fdb37122e925294a1294014
Aug 09 02:02:42 dist-test-slave-ss4n krb5kdc[5090](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240962, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.195@KRBTEST.COM for kudu/127.1.192.254@KRBTEST.COM
I20260809 02:02:43.004705 5187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:42.992549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.195:47889 (local address 127.1.192.254:39267)
0809 02:02:42.992730 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:42.992735 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:42.993782 (+ 1047us) server_negotiation.cc:407] Connection header received
0809 02:02:42.995448 (+ 1666us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:42.995451 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:42.995508 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:42.995592 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:42.996770 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:42.997313 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:42.998511 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:42.998735 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:42.998845 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:43.001260 (+ 2415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:02:43.001264 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:02:43.001267 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:02:43.001301 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:02:43.002667 (+ 1366us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:43.003177 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:43.003182 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:43.003184 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:43.003268 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:43.003609 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:43.003614 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:43.003616 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:43.003800 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:02:43.003911 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:43.004397 (+ 486us) server_negotiation.cc:299] Negotiation successful
0809 02:02:43.004527 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 02:02:43.005654 5661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:42.992982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:39267 (local address 127.1.192.195:47889)
0809 02:02:42.993588 (+ 606us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:42.993669 (+ 81us) client_negotiation.cc:174] Beginning negotiation
0809 02:02:42.995246 (+ 1577us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:02:42.995787 (+ 541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:02:42.995793 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:02:42.996153 (+ 360us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:02:42.996616 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:42.996629 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:42.997442 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:42.997446 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:42.998379 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:42.998389 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:42.998889 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:42.998893 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:42.999063 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:42.999628 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:02:42.999654 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:02:43.001101 (+ 1447us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:02:43.002787 (+ 1686us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:43.002792 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:43.002808 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:43.003067 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:43.003363 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:43.003367 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:43.003369 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:43.003511 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:43.003909 (+ 398us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:02:43.003916 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:02:43.004165 (+ 249us) client_negotiation.cc:769] Sending connection context
0809 02:02:43.004357 (+ 192us) client_negotiation.cc:240] Negotiation successful
0809 02:02:43.004691 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":150,"threads_started":1}
I20260809 02:02:43.006971 5657 heartbeater.cc:347] Connected to a master server at 127.1.192.254:39267
I20260809 02:02:43.007244 5657 heartbeater.cc:464] Registering TS with master...
I20260809 02:02:43.007861 5657 heartbeater.cc:511] Master 127.1.192.254:39267 requested a full tablet report, sending...
I20260809 02:02:43.008863 5140 ts_manager.cc:194] Registered new tserver with Master: 75c281901357485fae5835be2473f1a6 (127.1.192.195:42999)
I20260809 02:02:43.009455 5140 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.192.195@KRBTEST.COM'} at 127.1.192.195:47889
I20260809 02:02:43.019484 1795 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 02:02:43 dist-test-slave-ss4n krb5kdc[5090](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240963, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 09 02:02:43 dist-test-slave-ss4n krb5kdc[5090](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240963, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.1.192.254@KRBTEST.COM
I20260809 02:02:43.047906 5187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:43.038850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39340 (local address 127.1.192.254:39267)
0809 02:02:43.039060 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:43.039065 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:43.039156 (+ 91us) server_negotiation.cc:407] Connection header received
0809 02:02:43.039275 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:43.039277 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:43.039322 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:43.039397 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:43.040200 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:43.040802 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:43.041724 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:43.041945 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:43.042032 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:43.044661 (+ 2629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:02:43.044664 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:02:43.044666 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:02:43.044693 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:02:43.046221 (+ 1528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:43.046615 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:43.046617 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:43.046619 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:43.046741 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:43.046994 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:43.046996 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:43.046998 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:43.047244 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:02:43.047350 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:43.047555 (+ 205us) server_negotiation.cc:299] Negotiation successful
0809 02:02:43.047675 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
W20260809 02:02:43.048394 5140 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:39340
I20260809 02:02:43.049839 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 5213
I20260809 02:02:43.056829 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 5365
I20260809 02:02:43.067163 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 5513
I20260809 02:02:43.074561 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 5104
2026-08-09T02:02:43Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2331 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786240941968412-1795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:02:43 dist-test-slave-ss4n krb5kdc[5672](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:02:43 dist-test-slave-ss4n krb5kdc[5672](info): set up 2 sockets
Aug 09 02:02:43 dist-test-slave-ss4n krb5kdc[5672](info): commencing operation
krb5kdc: starting...
W20260809 02:02:44.211884 1795 mini_kdc.cc:121] Time spent starting KDC: real 1.114s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786240941968412-1795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786240941968412-1795-0/krb5kdc/test-user.keytab.
Aug 09 02:02:44 dist-test-slave-ss4n krb5kdc[5672](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240964, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:02:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:02:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.192.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.192.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.192.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab.
Entry for principal kudu/127.1.192.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab.
I20260809 02:02:44.346084 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786240941968412-1795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786240941968412-1795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786240941968412-1795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.192.254:32789
--webserver_interface=127.1.192.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab
--principal=kudu/127.1.192.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:42663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.192.254:32789
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:02:44.462492 5688 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:02:44.462826 5688 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:02:44.462869 5688 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:02:44.467043 5688 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:02:44.467120 5688 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:02:44.467141 5688 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:02:44.467159 5688 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:02:44.467175 5688 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:02:44.472815 5688 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:42663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786240941968412-1795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786240941968412-1795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.192.254:32789
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab
--principal=kudu/127.1.192.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.192.254:32789
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.192.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5688
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786240941968412-1795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:02:44.473987 5688 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:02:44.475037 5688 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:02:44.484531 5696 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:02:44.484547 5694 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:02:44.484549 5693 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:02:44.484838 5688 server_base.cc:1034] running on GCE node
I20260809 02:02:44.485448 5688 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:02:44.486624 5688 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:02:44.487833 5688 hybrid_clock.cc:648] HybridClock initialized: now 1786240964487824 us; error 30 us; skew 500 ppm
Aug 09 02:02:44 dist-test-slave-ss4n krb5kdc[5672](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240964, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:02:44.490756 5688 init.cc:377] Logged in from keytab as kudu/127.1.192.254@KRBTEST.COM (short username kudu)
I20260809 02:02:44.491722 5688 webserver.cc:460] Webserver started at http://127.1.192.254:37985/ using document root <none> and password file <none>
I20260809 02:02:44.492301 5688 fs_manager.cc:359] Metadata directory not provided
I20260809 02:02:44.492398 5688 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:02:44.492636 5688 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:02:44.494871 5688 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786240941968412-1795-0/minicluster-data/master-0/data/instance:
uuid: "8866c1044c934d109bc71dba530fdfac"
format_stamp: "Formatted at 2026-08-09 02:02:44 on dist-test-slave-ss4n"
server_key: "9420ef2e19635f209978944e6ecc9067"
server_key_iv: "992d828499d7931b949ae844acb61cbc"
server_key_version: "encryptionkey@0"
I20260809 02:02:44.495460 5688 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786240941968412-1795-0/minicluster-data/master-0/wal/instance:
uuid: "8866c1044c934d109bc71dba530fdfac"
format_stamp: "Formatted at 2026-08-09 02:02:44 on dist-test-slave-ss4n"
server_key: "9420ef2e19635f209978944e6ecc9067"
server_key_iv: "992d828499d7931b949ae844acb61cbc"
server_key_version: "encryptionkey@0"
I20260809 02:02:44.499425 5688 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 02:02:44.501955 5704 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:44.503257 5688 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 02:02:44.503407 5688 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786240941968412-1795-0/minicluster-data/master-0/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786240941968412-1795-0/minicluster-data/master-0/wal
uuid: "8866c1044c934d109bc71dba530fdfac"
format_stamp: "Formatted at 2026-08-09 02:02:44 on dist-test-slave-ss4n"
server_key: "9420ef2e19635f209978944e6ecc9067"
server_key_iv: "992d828499d7931b949ae844acb61cbc"
server_key_version: "encryptionkey@0"
I20260809 02:02:44.503530 5688 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786240941968412-1795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786240941968412-1795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786240941968412-1795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:02:44.516487 5688 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:02:44.519537 5688 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:02:44.519757 5688 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:02:44.527235 5688 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.254:32789
I20260809 02:02:44.527266 5764 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.254:32789 every 8 connection(s)
I20260809 02:02:44.528223 5688 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
I20260809 02:02:44.531299 5765 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:02:44.533887 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 5688
I20260809 02:02:44.534016 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786240941968412-1795-0/minicluster-data/master-0/wal/instance
I20260809 02:02:44.534325 1795 external_mini_cluster.cc:1428] Setting key be0ac5043349750ab352be6444e6ba4d
I20260809 02:02:44.537655 5765 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8866c1044c934d109bc71dba530fdfac: Bootstrap starting.
I20260809 02:02:44.539914 5765 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8866c1044c934d109bc71dba530fdfac: Neither blocks nor log segments found. Creating new log.
I20260809 02:02:44.540824 5765 log.cc:824] T 00000000000000000000000000000000 P 8866c1044c934d109bc71dba530fdfac: Log is configured to *not* fsync() on all Append() calls
I20260809 02:02:44.543402 5765 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8866c1044c934d109bc71dba530fdfac: No bootstrap required, opened a new log
Aug 09 02:02:44 dist-test-slave-ss4n krb5kdc[5672](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240964, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.192.254@KRBTEST.COM
I20260809 02:02:44.547948 5765 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8866c1044c934d109bc71dba530fdfac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8866c1044c934d109bc71dba530fdfac" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 32789 } }
I20260809 02:02:44.548261 5765 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8866c1044c934d109bc71dba530fdfac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:02:44.548327 5765 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8866c1044c934d109bc71dba530fdfac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8866c1044c934d109bc71dba530fdfac, State: Initialized, Role: FOLLOWER
I20260809 02:02:44.548842 5765 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8866c1044c934d109bc71dba530fdfac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8866c1044c934d109bc71dba530fdfac" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 32789 } }
I20260809 02:02:44.549034 5765 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8866c1044c934d109bc71dba530fdfac [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:02:44.549105 5765 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8866c1044c934d109bc71dba530fdfac [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:02:44.549196 5765 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8866c1044c934d109bc71dba530fdfac [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:02:44.550446 5768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:44.535844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38430 (local address 127.1.192.254:32789)
0809 02:02:44.536324 (+ 480us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:44.536339 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:44.536373 (+ 34us) server_negotiation.cc:407] Connection header received
0809 02:02:44.537003 (+ 630us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:44.537019 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:44.537355 (+ 336us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:44.537743 (+ 388us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:44.538566 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:44.539551 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:44.540495 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:44.541002 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:44.541213 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:44.544289 (+ 3076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:02:44.544295 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:02:44.544312 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:02:44.544344 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:02:44.546994 (+ 2650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:44.547497 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:44.547504 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:44.547510 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:44.547667 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:44.547951 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:44.547954 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:44.547957 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:44.548510 (+ 553us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:02:44.548710 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:44.549032 (+ 322us) server_negotiation.cc:299] Negotiation successful
0809 02:02:44.549368 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":338,"thread_start_us":149,"threads_started":1}
I20260809 02:02:44.550438 5765 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8866c1044c934d109bc71dba530fdfac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8866c1044c934d109bc71dba530fdfac" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 32789 } }
I20260809 02:02:44.550845 5765 leader_election.cc:302] T 00000000000000000000000000000000 P 8866c1044c934d109bc71dba530fdfac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8866c1044c934d109bc71dba530fdfac; no voters:
I20260809 02:02:44.551205 5765 leader_election.cc:288] T 00000000000000000000000000000000 P 8866c1044c934d109bc71dba530fdfac [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:02:44.551345 5770 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8866c1044c934d109bc71dba530fdfac [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:02:44.551575 5770 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8866c1044c934d109bc71dba530fdfac [term 1 LEADER]: Becoming Leader. State: Replica: 8866c1044c934d109bc71dba530fdfac, State: Running, Role: LEADER
I20260809 02:02:44.551896 5770 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8866c1044c934d109bc71dba530fdfac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8866c1044c934d109bc71dba530fdfac" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 32789 } }
I20260809 02:02:44.552506 5765 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8866c1044c934d109bc71dba530fdfac [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:02:44.553161 5771 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8866c1044c934d109bc71dba530fdfac [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8866c1044c934d109bc71dba530fdfac. Latest consensus state: current_term: 1 leader_uuid: "8866c1044c934d109bc71dba530fdfac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8866c1044c934d109bc71dba530fdfac" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 32789 } } }
I20260809 02:02:44.553278 5771 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8866c1044c934d109bc71dba530fdfac [sys.catalog]: This master's current role is: LEADER
I20260809 02:02:44.553963 5772 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8866c1044c934d109bc71dba530fdfac [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8866c1044c934d109bc71dba530fdfac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8866c1044c934d109bc71dba530fdfac" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 32789 } } }
I20260809 02:02:44.554060 5772 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8866c1044c934d109bc71dba530fdfac [sys.catalog]: This master's current role is: LEADER
I20260809 02:02:44.554351 5776 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:02:44.557160 5776 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:02:44.562690 5776 catalog_manager.cc:1347] Generated new cluster ID: 2b79fda7a3094319adc36321eacb526f
I20260809 02:02:44.562844 5776 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:02:44.575073 5776 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:02:44.576170 5776 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:02:44.588007 5776 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8866c1044c934d109bc71dba530fdfac: Generated new TSK 0
I20260809 02:02:44.588896 5776 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.192.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.192.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.192.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.193.keytab.
Entry for principal kudu/127.1.192.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.193.keytab.
I20260809 02:02:44.630393 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786240941968412-1795-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786240941968412-1795-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786240941968412-1795-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786240941968412-1795-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.192.193:0
--local_ip_for_outbound_sockets=127.1.192.193
--webserver_interface=127.1.192.193
--webserver_port=0
--tserver_master_addrs=127.1.192.254:32789
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.193.keytab
--principal=kudu/127.1.192.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:42663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:02:44.749962 5793 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:02:44.750250 5793 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:02:44.750295 5793 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:02:44.754674 5793 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:02:44.754779 5793 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:02:44.754876 5793 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.192.193
I20260809 02:02:44.760793 5793 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:42663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786240941968412-1795-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786240941968412-1795-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.193.keytab
--principal=kudu/127.1.192.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.192.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786240941968412-1795-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.192.193
--webserver_port=0
--tserver_master_addrs=127.1.192.254:32789
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5793
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.192.193
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786240941968412-1795-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:02:44.762004 5793 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:02:44.763073 5793 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:02:44.773190 5801 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:02:44.773167 5800 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:02:44.773175 5803 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:02:44.773422 5793 server_base.cc:1034] running on GCE node
I20260809 02:02:44.773978 5793 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:02:44.774699 5793 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:02:44.775890 5793 hybrid_clock.cc:648] HybridClock initialized: now 1786240964775861 us; error 50 us; skew 500 ppm
Aug 09 02:02:44 dist-test-slave-ss4n krb5kdc[5672](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240964, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:02:44.779598 5793 init.cc:377] Logged in from keytab as kudu/127.1.192.193@KRBTEST.COM (short username kudu)
I20260809 02:02:44.780633 5793 webserver.cc:460] Webserver started at http://127.1.192.193:42609/ using document root <none> and password file <none>
I20260809 02:02:44.781361 5793 fs_manager.cc:359] Metadata directory not provided
I20260809 02:02:44.781435 5793 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:02:44.781725 5793 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:02:44.783913 5793 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786240941968412-1795-0/minicluster-data/ts-0/data/instance:
uuid: "30703e0bd90649058baae8d75f488b6a"
format_stamp: "Formatted at 2026-08-09 02:02:44 on dist-test-slave-ss4n"
server_key: "7c111fe149a5e1ee3509672a5a56a46f"
server_key_iv: "d3710c48ae6ed772ae45295ad762a398"
server_key_version: "encryptionkey@0"
I20260809 02:02:44.784483 5793 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786240941968412-1795-0/minicluster-data/ts-0/wal/instance:
uuid: "30703e0bd90649058baae8d75f488b6a"
format_stamp: "Formatted at 2026-08-09 02:02:44 on dist-test-slave-ss4n"
server_key: "7c111fe149a5e1ee3509672a5a56a46f"
server_key_iv: "d3710c48ae6ed772ae45295ad762a398"
server_key_version: "encryptionkey@0"
I20260809 02:02:44.788604 5793 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260809 02:02:44.791340 5810 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:44.792610 5793 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 02:02:44.792760 5793 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786240941968412-1795-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786240941968412-1795-0/minicluster-data/ts-0/wal
uuid: "30703e0bd90649058baae8d75f488b6a"
format_stamp: "Formatted at 2026-08-09 02:02:44 on dist-test-slave-ss4n"
server_key: "7c111fe149a5e1ee3509672a5a56a46f"
server_key_iv: "d3710c48ae6ed772ae45295ad762a398"
server_key_version: "encryptionkey@0"
I20260809 02:02:44.792881 5793 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786240941968412-1795-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786240941968412-1795-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786240941968412-1795-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:02:44.803028 5793 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:02:44.805998 5793 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:02:44.806213 5793 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:02:44.806902 5793 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:02:44.807827 5793 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:02:44.807874 5793 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:44.807935 5793 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:02:44.807966 5793 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:44.817830 5793 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.193:36531
I20260809 02:02:44.817891 5939 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.193:36531 every 8 connection(s)
I20260809 02:02:44.818811 5793 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786240941968412-1795-0/minicluster-data/ts-0/data/info.pb
I20260809 02:02:44.828295 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 5793
I20260809 02:02:44.828408 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786240941968412-1795-0/minicluster-data/ts-0/wal/instance
I20260809 02:02:44.828749 1795 external_mini_cluster.cc:1428] Setting key 563b35cb638fcbc41f234d00707c8e45
Aug 09 02:02:44 dist-test-slave-ss4n krb5kdc[5672](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240964, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.193@KRBTEST.COM for kudu/127.1.192.254@KRBTEST.COM
I20260809 02:02:44.834322 5768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:44.820823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.193:59349 (local address 127.1.192.254:32789)
0809 02:02:44.821002 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:44.821007 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:44.821845 (+ 838us) server_negotiation.cc:407] Connection header received
0809 02:02:44.823228 (+ 1383us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:44.823231 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:44.823281 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:44.823362 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:44.824541 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:44.825085 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:44.826239 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:44.826476 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:44.826564 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:44.829783 (+ 3219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:02:44.829787 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:02:44.829789 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:02:44.829815 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:02:44.831895 (+ 2080us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:44.832475 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:44.832490 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:44.832495 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:44.832611 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:44.833094 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:44.833096 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:44.833098 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:44.833279 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:02:44.833354 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:44.834085 (+ 731us) server_negotiation.cc:299] Negotiation successful
0809 02:02:44.834196 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 02:02:44.835605 5942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:44.821169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:32789 (local address 127.1.192.193:59349)
0809 02:02:44.821694 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:44.821741 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 02:02:44.823014 (+ 1273us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:02:44.823526 (+ 512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:02:44.823532 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:02:44.823918 (+ 386us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:02:44.824390 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:44.824403 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:44.825211 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:44.825214 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:44.826123 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:44.826132 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:44.826658 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:44.826662 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:44.826848 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:44.827631 (+ 783us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:02:44.827649 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:02:44.829566 (+ 1917us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:02:44.832031 (+ 2465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:44.832036 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:44.832052 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:44.832347 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:44.832744 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:44.832749 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:44.832751 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:44.832983 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:44.833389 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:02:44.833398 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:02:44.833782 (+ 384us) client_negotiation.cc:769] Sending connection context
0809 02:02:44.834011 (+ 229us) client_negotiation.cc:240] Negotiation successful
0809 02:02:44.834422 (+ 411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":112,"threads_started":1}
I20260809 02:02:44.837096 5940 heartbeater.cc:347] Connected to a master server at 127.1.192.254:32789
I20260809 02:02:44.837400 5940 heartbeater.cc:464] Registering TS with master...
I20260809 02:02:44.838129 5940 heartbeater.cc:511] Master 127.1.192.254:32789 requested a full tablet report, sending...
I20260809 02:02:44.840055 5723 ts_manager.cc:194] Registered new tserver with Master: 30703e0bd90649058baae8d75f488b6a (127.1.192.193:36531)
WARNING: no policy specified for kudu/127.1.192.194@KRBTEST.COM; defaulting to no policy
I20260809 02:02:44.841354 5723 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.192.193@KRBTEST.COM'} at 127.1.192.193:59349
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.192.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.192.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.194.keytab.
Entry for principal kudu/127.1.192.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.194.keytab.
I20260809 02:02:44.861955 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786240941968412-1795-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786240941968412-1795-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786240941968412-1795-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786240941968412-1795-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.192.194:0
--local_ip_for_outbound_sockets=127.1.192.194
--webserver_interface=127.1.192.194
--webserver_port=0
--tserver_master_addrs=127.1.192.254:32789
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.194.keytab
--principal=kudu/127.1.192.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:42663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:02:44.975519 5946 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:02:44.975829 5946 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:02:44.975914 5946 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:02:44.980010 5946 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:02:44.980098 5946 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:02:44.980186 5946 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.192.194
I20260809 02:02:44.985569 5946 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:42663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786240941968412-1795-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786240941968412-1795-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.194.keytab
--principal=kudu/127.1.192.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.192.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786240941968412-1795-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.192.194
--webserver_port=0
--tserver_master_addrs=127.1.192.254:32789
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5946
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.192.194
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786240941968412-1795-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:02:44.986685 5946 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:02:44.987736 5946 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:02:44.997763 5951 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:02:44.997735 5952 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:02:44.997897 5946 server_base.cc:1034] running on GCE node
W20260809 02:02:44.997759 5956 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:02:44.998605 5946 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:02:44.999248 5946 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:02:45.000416 5946 hybrid_clock.cc:648] HybridClock initialized: now 1786240965000396 us; error 32 us; skew 500 ppm
Aug 09 02:02:45 dist-test-slave-ss4n krb5kdc[5672](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240965, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:02:45.003337 5946 init.cc:377] Logged in from keytab as kudu/127.1.192.194@KRBTEST.COM (short username kudu)
I20260809 02:02:45.004257 5946 webserver.cc:460] Webserver started at http://127.1.192.194:43749/ using document root <none> and password file <none>
I20260809 02:02:45.004823 5946 fs_manager.cc:359] Metadata directory not provided
I20260809 02:02:45.004884 5946 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:02:45.005119 5946 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:02:45.007333 5946 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786240941968412-1795-0/minicluster-data/ts-1/data/instance:
uuid: "c7b2743f81244a6ca044da5ee495cf03"
format_stamp: "Formatted at 2026-08-09 02:02:45 on dist-test-slave-ss4n"
server_key: "87400ac6448dc0874a6b54ac372c4a8c"
server_key_iv: "1f72fe895245647a3908d2b46b04a80c"
server_key_version: "encryptionkey@0"
I20260809 02:02:45.007947 5946 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786240941968412-1795-0/minicluster-data/ts-1/wal/instance:
uuid: "c7b2743f81244a6ca044da5ee495cf03"
format_stamp: "Formatted at 2026-08-09 02:02:45 on dist-test-slave-ss4n"
server_key: "87400ac6448dc0874a6b54ac372c4a8c"
server_key_iv: "1f72fe895245647a3908d2b46b04a80c"
server_key_version: "encryptionkey@0"
I20260809 02:02:45.011952 5946 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 02:02:45.014459 5963 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:45.015753 5946 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 02:02:45.015898 5946 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786240941968412-1795-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786240941968412-1795-0/minicluster-data/ts-1/wal
uuid: "c7b2743f81244a6ca044da5ee495cf03"
format_stamp: "Formatted at 2026-08-09 02:02:45 on dist-test-slave-ss4n"
server_key: "87400ac6448dc0874a6b54ac372c4a8c"
server_key_iv: "1f72fe895245647a3908d2b46b04a80c"
server_key_version: "encryptionkey@0"
I20260809 02:02:45.016021 5946 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786240941968412-1795-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786240941968412-1795-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786240941968412-1795-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:02:45.051302 5946 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:02:45.055146 5946 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:02:45.055375 5946 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:02:45.056020 5946 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:02:45.056910 5946 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:02:45.056985 5946 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:45.057063 5946 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:02:45.057118 5946 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:45.067216 5946 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.194:43007
I20260809 02:02:45.067242 6086 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.194:43007 every 8 connection(s)
I20260809 02:02:45.068231 5946 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786240941968412-1795-0/minicluster-data/ts-1/data/info.pb
I20260809 02:02:45.069808 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 5946
I20260809 02:02:45.069952 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786240941968412-1795-0/minicluster-data/ts-1/wal/instance
I20260809 02:02:45.070327 1795 external_mini_cluster.cc:1428] Setting key ad6a20ec6ea7eaad60417e861d0660a6
Aug 09 02:02:45 dist-test-slave-ss4n krb5kdc[5672](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240965, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.194@KRBTEST.COM for kudu/127.1.192.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.192.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.192.195@KRBTEST.COM" created.
I20260809 02:02:45.084506 5768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:45.070300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.194:44119 (local address 127.1.192.254:32789)
0809 02:02:45.070467 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:45.070473 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:45.071515 (+ 1042us) server_negotiation.cc:407] Connection header received
0809 02:02:45.073080 (+ 1565us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:45.073087 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:45.073158 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:45.073260 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:45.074567 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:45.075190 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:45.076472 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:45.076822 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:45.076953 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:45.080015 (+ 3062us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:02:45.080020 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:02:45.080022 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:02:45.080058 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:02:45.082025 (+ 1967us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:45.082663 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:45.082667 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:45.082669 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:45.082763 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:45.083159 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:45.083165 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:45.083167 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:45.083382 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:02:45.083503 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:45.084150 (+ 647us) server_negotiation.cc:299] Negotiation successful
0809 02:02:45.084283 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 02:02:45.085561 6089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:45.070771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:32789 (local address 127.1.192.194:44119)
0809 02:02:45.071334 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:45.071392 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0809 02:02:45.072843 (+ 1451us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:02:45.073404 (+ 561us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:02:45.073411 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:02:45.073864 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:02:45.074402 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:45.074417 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:45.075341 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:45.075346 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:45.076324 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:45.076333 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:45.077002 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:45.077007 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:45.077200 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:45.077926 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:02:45.077943 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:02:45.079815 (+ 1872us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:02:45.082204 (+ 2389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:45.082211 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:45.082229 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:45.082522 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:45.082870 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:45.082874 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:45.082876 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:45.083040 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:45.083501 (+ 461us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:02:45.083509 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:02:45.083830 (+ 321us) client_negotiation.cc:769] Sending connection context
0809 02:02:45.084072 (+ 242us) client_negotiation.cc:240] Negotiation successful
0809 02:02:45.084474 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":172,"threads_started":1}
I20260809 02:02:45.086894 6087 heartbeater.cc:347] Connected to a master server at 127.1.192.254:32789
I20260809 02:02:45.087212 6087 heartbeater.cc:464] Registering TS with master...
I20260809 02:02:45.087901 6087 heartbeater.cc:511] Master 127.1.192.254:32789 requested a full tablet report, sending...
I20260809 02:02:45.089176 5723 ts_manager.cc:194] Registered new tserver with Master: c7b2743f81244a6ca044da5ee495cf03 (127.1.192.194:43007)
I20260809 02:02:45.089920 5723 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.192.194@KRBTEST.COM'} at 127.1.192.194:44119
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.192.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.195.keytab.
Entry for principal kudu/127.1.192.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.195.keytab.
I20260809 02:02:45.104803 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786240941968412-1795-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786240941968412-1795-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786240941968412-1795-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786240941968412-1795-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.192.195:0
--local_ip_for_outbound_sockets=127.1.192.195
--webserver_interface=127.1.192.195
--webserver_port=0
--tserver_master_addrs=127.1.192.254:32789
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.195.keytab
--principal=kudu/127.1.192.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:42663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:02:45.218914 6092 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:02:45.219254 6092 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:02:45.219332 6092 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:02:45.223589 6092 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:02:45.223655 6092 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:02:45.223716 6092 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.192.195
I20260809 02:02:45.229233 6092 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:42663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786240941968412-1795-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786240941968412-1795-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.195.keytab
--principal=kudu/127.1.192.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.192.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786240941968412-1795-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.192.195
--webserver_port=0
--tserver_master_addrs=127.1.192.254:32789
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6092
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.192.195
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786240941968412-1795-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:02:45.230472 6092 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:02:45.231647 6092 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:02:45.241302 6098 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:02:45.241308 6100 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:02:45.241380 6097 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:02:45.241505 6092 server_base.cc:1034] running on GCE node
I20260809 02:02:45.241994 6092 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:02:45.242710 6092 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:02:45.243894 6092 hybrid_clock.cc:648] HybridClock initialized: now 1786240965243878 us; error 30 us; skew 500 ppm
Aug 09 02:02:45 dist-test-slave-ss4n krb5kdc[5672](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240965, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:02:45.246814 6092 init.cc:377] Logged in from keytab as kudu/127.1.192.195@KRBTEST.COM (short username kudu)
I20260809 02:02:45.247651 6092 webserver.cc:460] Webserver started at http://127.1.192.195:37419/ using document root <none> and password file <none>
I20260809 02:02:45.248171 6092 fs_manager.cc:359] Metadata directory not provided
I20260809 02:02:45.248229 6092 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:02:45.248387 6092 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:02:45.250432 6092 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786240941968412-1795-0/minicluster-data/ts-2/data/instance:
uuid: "6e9e342648ad47aaa97f306ed9ef28f8"
format_stamp: "Formatted at 2026-08-09 02:02:45 on dist-test-slave-ss4n"
server_key: "b629059c769972dc3a8291ffbc18da22"
server_key_iv: "c41d8ee252a3c42795b30881eaab20a8"
server_key_version: "encryptionkey@0"
I20260809 02:02:45.250937 6092 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786240941968412-1795-0/minicluster-data/ts-2/wal/instance:
uuid: "6e9e342648ad47aaa97f306ed9ef28f8"
format_stamp: "Formatted at 2026-08-09 02:02:45 on dist-test-slave-ss4n"
server_key: "b629059c769972dc3a8291ffbc18da22"
server_key_iv: "c41d8ee252a3c42795b30881eaab20a8"
server_key_version: "encryptionkey@0"
I20260809 02:02:45.255095 6092 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 02:02:45.257596 6107 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:45.258905 6092 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 02:02:45.259006 6092 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786240941968412-1795-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786240941968412-1795-0/minicluster-data/ts-2/wal
uuid: "6e9e342648ad47aaa97f306ed9ef28f8"
format_stamp: "Formatted at 2026-08-09 02:02:45 on dist-test-slave-ss4n"
server_key: "b629059c769972dc3a8291ffbc18da22"
server_key_iv: "c41d8ee252a3c42795b30881eaab20a8"
server_key_version: "encryptionkey@0"
I20260809 02:02:45.259097 6092 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786240941968412-1795-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786240941968412-1795-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786240941968412-1795-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:02:45.280251 6092 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:02:45.283603 6092 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:02:45.283811 6092 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:02:45.284382 6092 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:02:45.285205 6092 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:02:45.285256 6092 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:45.285298 6092 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:02:45.285315 6092 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:45.295286 6092 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.195:36721
I20260809 02:02:45.295308 6233 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.195:36721 every 8 connection(s)
I20260809 02:02:45.296448 6092 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786240941968412-1795-0/minicluster-data/ts-2/data/info.pb
I20260809 02:02:45.302500 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 6092
I20260809 02:02:45.302685 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786240941968412-1795-0/minicluster-data/ts-2/wal/instance
I20260809 02:02:45.302989 1795 external_mini_cluster.cc:1428] Setting key 9c032fb65cb358f610a8bbd59632f008
Aug 09 02:02:45 dist-test-slave-ss4n krb5kdc[5672](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240965, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.195@KRBTEST.COM for kudu/127.1.192.254@KRBTEST.COM
I20260809 02:02:45.310536 5768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:45.298440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.195:58221 (local address 127.1.192.254:32789)
0809 02:02:45.298584 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:45.298588 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:45.299433 (+ 845us) server_negotiation.cc:407] Connection header received
0809 02:02:45.300794 (+ 1361us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:45.300797 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:45.300845 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:45.300924 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:45.302232 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:45.303077 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:45.304281 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:45.304509 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:45.304628 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:45.307245 (+ 2617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:02:45.307248 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:02:45.307250 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:02:45.307274 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:02:45.308655 (+ 1381us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:45.309125 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:45.309127 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:45.309129 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:45.309201 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:45.309546 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:45.309553 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:45.309554 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:45.309765 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:02:45.309846 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:45.310310 (+ 464us) server_negotiation.cc:299] Negotiation successful
0809 02:02:45.310409 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 02:02:45.311412 6238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:45.298830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:32789 (local address 127.1.192.195:58221)
0809 02:02:45.299273 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:45.299317 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 02:02:45.300566 (+ 1249us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:02:45.301081 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:02:45.301087 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:02:45.301443 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:02:45.302017 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:45.302030 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:45.303231 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:45.303234 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:45.304151 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:45.304160 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:45.304678 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:45.304683 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:45.304853 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:45.305523 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:02:45.305540 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:02:45.307085 (+ 1545us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:02:45.308759 (+ 1674us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:45.308764 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:45.308775 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:45.309036 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:45.309283 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:45.309285 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:45.309287 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:45.309418 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:45.309858 (+ 440us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:02:45.309864 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:02:45.310097 (+ 233us) client_negotiation.cc:769] Sending connection context
0809 02:02:45.310265 (+ 168us) client_negotiation.cc:240] Negotiation successful
0809 02:02:45.310522 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":122,"threads_started":1}
I20260809 02:02:45.312580 6234 heartbeater.cc:347] Connected to a master server at 127.1.192.254:32789
I20260809 02:02:45.312831 6234 heartbeater.cc:464] Registering TS with master...
I20260809 02:02:45.313328 6234 heartbeater.cc:511] Master 127.1.192.254:32789 requested a full tablet report, sending...
I20260809 02:02:45.314407 5723 ts_manager.cc:194] Registered new tserver with Master: 6e9e342648ad47aaa97f306ed9ef28f8 (127.1.192.195:36721)
I20260809 02:02:45.314985 5723 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.192.195@KRBTEST.COM'} at 127.1.192.195:58221
I20260809 02:02:45.316923 1795 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 02:02:45 dist-test-slave-ss4n krb5kdc[5672](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240965, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 02:02:45 dist-test-slave-ss4n krb5kdc[5672](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240965, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.192.193@KRBTEST.COM
I20260809 02:02:45.346354 6243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:45.336313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46922 (local address 127.1.192.193:36531)
0809 02:02:45.336627 (+ 314us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:45.336634 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:45.336669 (+ 35us) server_negotiation.cc:407] Connection header received
0809 02:02:45.336780 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:45.336785 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:45.336980 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:45.337107 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:45.337878 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:45.338647 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:45.339450 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:45.339846 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:45.339929 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:45.342502 (+ 2573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:02:45.342514 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:02:45.342528 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:02:45.342553 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:02:45.344360 (+ 1807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:45.344848 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:45.344854 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:45.344856 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:45.344957 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:45.345263 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:45.345265 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:45.345266 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:45.345606 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:02:45.345784 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:45.345955 (+ 171us) server_negotiation.cc:299] Negotiation successful
0809 02:02:45.346131 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":212,"thread_start_us":92,"threads_started":1}
Aug 09 02:02:45 dist-test-slave-ss4n krb5kdc[5672](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240965, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.192.254@KRBTEST.COM
I20260809 02:02:45.358088 5768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:45.349348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38442 (local address 127.1.192.254:32789)
0809 02:02:45.349520 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:45.349527 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:45.349551 (+ 24us) server_negotiation.cc:407] Connection header received
0809 02:02:45.349695 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:45.349697 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:45.349742 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:45.349808 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:45.350419 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:45.350953 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:45.351721 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:45.351924 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:45.351999 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:45.354608 (+ 2609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:02:45.354612 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:02:45.354614 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:02:45.354646 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:02:45.356159 (+ 1513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:45.356638 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:45.356651 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:45.356653 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:45.356734 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:45.357053 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:45.357056 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:45.357057 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:45.357291 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:02:45.357400 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:45.357748 (+ 348us) server_negotiation.cc:299] Negotiation successful
0809 02:02:45.357859 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
W20260809 02:02:45.358564 5723 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38442
I20260809 02:02:45.359318 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 5793
I20260809 02:02:45.369206 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 5946
I20260809 02:02:45.376343 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 6092
I20260809 02:02:45.382661 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 5688
2026-08-09T02:02:45Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2308 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786240941968412-1795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:02:45 dist-test-slave-ss4n krb5kdc[6248](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:02:45 dist-test-slave-ss4n krb5kdc[6248](info): set up 2 sockets
Aug 09 02:02:45 dist-test-slave-ss4n krb5kdc[6248](info): commencing operation
krb5kdc: starting...
W20260809 02:02:46.600508 1795 mini_kdc.cc:121] Time spent starting KDC: real 1.194s user 0.005s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
W20260809 02:02:46.722359 1795 mini_kdc.cc:253] Time spent creating user principal test-admin: real 0.122s user 0.000s sys 0.002s
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
W20260809 02:02:46.886612 1795 mini_kdc.cc:253] Time spent creating user principal test-user: real 0.164s user 0.000s sys 0.002s
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
W20260809 02:02:47.003357 1795 mini_kdc.cc:253] Time spent creating user principal joe-interloper: real 0.116s user 0.000s sys 0.003s
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786240941968412-1795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786240941968412-1795-0/krb5kdc/test-user.keytab.
W20260809 02:02:47.336301 1795 mini_kdc.cc:302] Time spent creating keytab for test-user: real 0.333s user 0.000s sys 0.003s
Aug 09 02:02:47 dist-test-slave-ss4n krb5kdc[6248](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240967, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:02:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:02:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.192.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.192.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.192.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab.
Entry for principal kudu/127.1.192.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab.
I20260809 02:02:47.391103 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786240941968412-1795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786240941968412-1795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786240941968412-1795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.192.254:35875
--webserver_interface=127.1.192.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab
--principal=kudu/127.1.192.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:44077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.192.254:35875
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:02:47.505863 6264 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:02:47.506193 6264 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:02:47.506278 6264 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:02:47.510402 6264 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:02:47.510512 6264 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:02:47.510547 6264 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:02:47.510591 6264 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:02:47.510623 6264 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:02:47.516113 6264 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:44077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786240941968412-1795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786240941968412-1795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.192.254:35875
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab
--principal=kudu/127.1.192.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.192.254:35875
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.192.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6264
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786240941968412-1795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:02:47.517318 6264 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:02:47.518424 6264 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:02:47.526955 6272 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:02:47.526955 6269 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:02:47.526964 6270 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:02:47.527267 6264 server_base.cc:1034] running on GCE node
I20260809 02:02:47.528090 6264 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:02:47.529269 6264 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:02:47.530427 6264 hybrid_clock.cc:648] HybridClock initialized: now 1786240967530431 us; error 53 us; skew 500 ppm
Aug 09 02:02:47 dist-test-slave-ss4n krb5kdc[6248](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240967, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:02:47.537159 6264 init.cc:377] Logged in from keytab as kudu/127.1.192.254@KRBTEST.COM (short username kudu)
I20260809 02:02:47.538147 6264 fs_manager.cc:359] Metadata directory not provided
I20260809 02:02:47.538219 6264 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:02:47.538448 6264 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:02:47.540654 6264 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786240941968412-1795-0/minicluster-data/master-0/data/instance:
uuid: "8e9ac229743b44908e5b8a96323f8ca1"
format_stamp: "Formatted at 2026-08-09 02:02:47 on dist-test-slave-ss4n"
server_key: "4baf91f1a6feb77749402449f9bcdd38"
server_key_iv: "81788c4f3a6d9d3d23a4624b9fe16f9b"
server_key_version: "encryptionkey@0"
I20260809 02:02:47.541285 6264 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786240941968412-1795-0/minicluster-data/master-0/wal/instance:
uuid: "8e9ac229743b44908e5b8a96323f8ca1"
format_stamp: "Formatted at 2026-08-09 02:02:47 on dist-test-slave-ss4n"
server_key: "4baf91f1a6feb77749402449f9bcdd38"
server_key_iv: "81788c4f3a6d9d3d23a4624b9fe16f9b"
server_key_version: "encryptionkey@0"
I20260809 02:02:47.545209 6264 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.003s
I20260809 02:02:47.547772 6278 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:47.549144 6264 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 02:02:47.549263 6264 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786240941968412-1795-0/minicluster-data/master-0/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786240941968412-1795-0/minicluster-data/master-0/wal
uuid: "8e9ac229743b44908e5b8a96323f8ca1"
format_stamp: "Formatted at 2026-08-09 02:02:47 on dist-test-slave-ss4n"
server_key: "4baf91f1a6feb77749402449f9bcdd38"
server_key_iv: "81788c4f3a6d9d3d23a4624b9fe16f9b"
server_key_version: "encryptionkey@0"
I20260809 02:02:47.549361 6264 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786240941968412-1795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786240941968412-1795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786240941968412-1795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:02:47.592448 6264 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:02:47.595783 6264 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:02:47.595953 6264 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:02:47.603294 6264 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.254:35875
I20260809 02:02:47.603576 6334 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.254:35875 every 8 connection(s)
I20260809 02:02:47.604032 6264 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
I20260809 02:02:47.606859 6335 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:02:47.609267 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 6264
I20260809 02:02:47.609376 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786240941968412-1795-0/minicluster-data/master-0/wal/instance
I20260809 02:02:47.609709 1795 external_mini_cluster.cc:1428] Setting key 6185bbdb8cd49d5d636a0e63d396f712
I20260809 02:02:47.612959 6335 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e9ac229743b44908e5b8a96323f8ca1: Bootstrap starting.
I20260809 02:02:47.615305 6335 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8e9ac229743b44908e5b8a96323f8ca1: Neither blocks nor log segments found. Creating new log.
I20260809 02:02:47.616096 6335 log.cc:824] T 00000000000000000000000000000000 P 8e9ac229743b44908e5b8a96323f8ca1: Log is configured to *not* fsync() on all Append() calls
I20260809 02:02:47.618202 6335 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e9ac229743b44908e5b8a96323f8ca1: No bootstrap required, opened a new log
Aug 09 02:02:47 dist-test-slave-ss4n krb5kdc[6248](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240967, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.192.254@KRBTEST.COM
I20260809 02:02:47.622242 6335 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8e9ac229743b44908e5b8a96323f8ca1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e9ac229743b44908e5b8a96323f8ca1" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 35875 } }
I20260809 02:02:47.622525 6335 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8e9ac229743b44908e5b8a96323f8ca1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:02:47.622613 6335 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8e9ac229743b44908e5b8a96323f8ca1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e9ac229743b44908e5b8a96323f8ca1, State: Initialized, Role: FOLLOWER
I20260809 02:02:47.623122 6335 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8e9ac229743b44908e5b8a96323f8ca1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e9ac229743b44908e5b8a96323f8ca1" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 35875 } }
I20260809 02:02:47.623287 6335 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8e9ac229743b44908e5b8a96323f8ca1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:02:47.623375 6335 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8e9ac229743b44908e5b8a96323f8ca1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:02:47.623524 6335 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8e9ac229743b44908e5b8a96323f8ca1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:02:47.624480 6338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:47.611183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60728 (local address 127.1.192.254:35875)
0809 02:02:47.611674 (+ 491us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:47.611690 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:47.611737 (+ 47us) server_negotiation.cc:407] Connection header received
0809 02:02:47.612370 (+ 633us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:47.612387 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:47.612716 (+ 329us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:47.613065 (+ 349us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:47.613930 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:47.614931 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:47.615889 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:47.616433 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:47.616611 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:47.619303 (+ 2692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:02:47.619314 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:02:47.619334 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:02:47.619363 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:02:47.621219 (+ 1856us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:47.621778 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:47.621787 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:47.621798 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:47.621919 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:47.622243 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:47.622246 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:47.622248 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:47.622796 (+ 548us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:02:47.623020 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:47.623321 (+ 301us) server_negotiation.cc:299] Negotiation successful
0809 02:02:47.623677 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":143,"threads_started":1}
I20260809 02:02:47.624567 6335 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8e9ac229743b44908e5b8a96323f8ca1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e9ac229743b44908e5b8a96323f8ca1" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 35875 } }
I20260809 02:02:47.624979 6335 leader_election.cc:302] T 00000000000000000000000000000000 P 8e9ac229743b44908e5b8a96323f8ca1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8e9ac229743b44908e5b8a96323f8ca1; no voters:
I20260809 02:02:47.625360 6335 leader_election.cc:288] T 00000000000000000000000000000000 P 8e9ac229743b44908e5b8a96323f8ca1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:02:47.625450 6340 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8e9ac229743b44908e5b8a96323f8ca1 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:02:47.625679 6340 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8e9ac229743b44908e5b8a96323f8ca1 [term 1 LEADER]: Becoming Leader. State: Replica: 8e9ac229743b44908e5b8a96323f8ca1, State: Running, Role: LEADER
I20260809 02:02:47.626042 6340 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8e9ac229743b44908e5b8a96323f8ca1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e9ac229743b44908e5b8a96323f8ca1" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 35875 } }
I20260809 02:02:47.626718 6335 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8e9ac229743b44908e5b8a96323f8ca1 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:02:47.627952 6341 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e9ac229743b44908e5b8a96323f8ca1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8e9ac229743b44908e5b8a96323f8ca1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e9ac229743b44908e5b8a96323f8ca1" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 35875 } } }
I20260809 02:02:47.628082 6341 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e9ac229743b44908e5b8a96323f8ca1 [sys.catalog]: This master's current role is: LEADER
I20260809 02:02:47.628440 6342 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e9ac229743b44908e5b8a96323f8ca1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8e9ac229743b44908e5b8a96323f8ca1. Latest consensus state: current_term: 1 leader_uuid: "8e9ac229743b44908e5b8a96323f8ca1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e9ac229743b44908e5b8a96323f8ca1" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 35875 } } }
I20260809 02:02:47.628539 6349 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:02:47.629141 6342 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e9ac229743b44908e5b8a96323f8ca1 [sys.catalog]: This master's current role is: LEADER
I20260809 02:02:47.631747 6349 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:02:47.637837 6349 catalog_manager.cc:1347] Generated new cluster ID: ccbd66d49bce476f8b79226122c094cb
I20260809 02:02:47.637920 6349 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:02:47.654017 6349 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:02:47.655395 6349 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:02:47.675673 6349 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8e9ac229743b44908e5b8a96323f8ca1: Generated new TSK 0
I20260809 02:02:47.676621 6349 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.192.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.192.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.192.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.193.keytab.
Entry for principal kudu/127.1.192.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.193.keytab.
I20260809 02:02:47.718206 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786240941968412-1795-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786240941968412-1795-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786240941968412-1795-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786240941968412-1795-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.192.193:0
--local_ip_for_outbound_sockets=127.1.192.193
--webserver_interface=127.1.192.193
--webserver_port=0
--tserver_master_addrs=127.1.192.254:35875
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.193.keytab
--principal=kudu/127.1.192.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:44077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:02:47.835068 6366 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:02:47.835343 6366 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:02:47.835390 6366 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:02:47.839597 6366 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:02:47.839677 6366 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:02:47.839767 6366 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.192.193
I20260809 02:02:47.845387 6366 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:44077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786240941968412-1795-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786240941968412-1795-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.193.keytab
--principal=kudu/127.1.192.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.192.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786240941968412-1795-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.192.193
--webserver_port=0
--tserver_master_addrs=127.1.192.254:35875
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6366
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.192.193
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786240941968412-1795-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:02:47.846549 6366 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:02:47.847818 6366 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:02:47.857589 6371 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:02:47.857592 6372 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:02:47.857610 6374 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:02:47.857654 6366 server_base.cc:1034] running on GCE node
I20260809 02:02:47.858331 6366 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:02:47.858953 6366 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:02:47.860164 6366 hybrid_clock.cc:648] HybridClock initialized: now 1786240967860134 us; error 38 us; skew 500 ppm
Aug 09 02:02:47 dist-test-slave-ss4n krb5kdc[6248](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240967, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:02:47.863436 6366 init.cc:377] Logged in from keytab as kudu/127.1.192.193@KRBTEST.COM (short username kudu)
I20260809 02:02:47.864264 6366 fs_manager.cc:359] Metadata directory not provided
I20260809 02:02:47.864377 6366 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:02:47.864638 6366 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:02:47.867000 6366 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786240941968412-1795-0/minicluster-data/ts-0/data/instance:
uuid: "fc036ebbd7964e0e93de22efe4ffc180"
format_stamp: "Formatted at 2026-08-09 02:02:47 on dist-test-slave-ss4n"
server_key: "e6b4182c5ac146b6e95dd79001b0bd2a"
server_key_iv: "932f83f747111091cb3961d180f261cb"
server_key_version: "encryptionkey@0"
I20260809 02:02:47.867652 6366 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786240941968412-1795-0/minicluster-data/ts-0/wal/instance:
uuid: "fc036ebbd7964e0e93de22efe4ffc180"
format_stamp: "Formatted at 2026-08-09 02:02:47 on dist-test-slave-ss4n"
server_key: "e6b4182c5ac146b6e95dd79001b0bd2a"
server_key_iv: "932f83f747111091cb3961d180f261cb"
server_key_version: "encryptionkey@0"
I20260809 02:02:47.871747 6366 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 02:02:47.874563 6381 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:47.875842 6366 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 02:02:47.875986 6366 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786240941968412-1795-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786240941968412-1795-0/minicluster-data/ts-0/wal
uuid: "fc036ebbd7964e0e93de22efe4ffc180"
format_stamp: "Formatted at 2026-08-09 02:02:47 on dist-test-slave-ss4n"
server_key: "e6b4182c5ac146b6e95dd79001b0bd2a"
server_key_iv: "932f83f747111091cb3961d180f261cb"
server_key_version: "encryptionkey@0"
I20260809 02:02:47.876104 6366 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786240941968412-1795-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786240941968412-1795-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786240941968412-1795-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:02:47.911793 6366 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:02:47.916427 6366 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:02:47.916684 6366 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:02:47.917596 6366 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:02:47.918853 6366 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:02:47.918917 6366 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:47.918969 6366 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:02:47.919001 6366 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:47.930164 6366 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.193:39963
I20260809 02:02:47.930287 6506 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.193:39963 every 8 connection(s)
I20260809 02:02:47.930754 6366 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786240941968412-1795-0/minicluster-data/ts-0/data/info.pb
I20260809 02:02:47.936201 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 6366
I20260809 02:02:47.936352 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786240941968412-1795-0/minicluster-data/ts-0/wal/instance
I20260809 02:02:47.936636 1795 external_mini_cluster.cc:1428] Setting key cc9e320670eb6c9cc377fdba2b9a9700
Aug 09 02:02:47 dist-test-slave-ss4n krb5kdc[6248](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240967, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.193@KRBTEST.COM for kudu/127.1.192.254@KRBTEST.COM
I20260809 02:02:47.946329 6338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:47.932573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.193:38329 (local address 127.1.192.254:35875)
0809 02:02:47.932723 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:47.932727 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:47.933519 (+ 792us) server_negotiation.cc:407] Connection header received
0809 02:02:47.934910 (+ 1391us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:47.934914 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:47.934976 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:47.935087 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:47.936274 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:47.936925 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:47.938629 (+ 1704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:47.938858 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:47.938944 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:47.942042 (+ 3098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:02:47.942048 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:02:47.942054 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:02:47.942088 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:02:47.943845 (+ 1757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:47.944451 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:47.944457 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:47.944461 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:47.944562 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:47.944949 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:47.944956 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:47.944959 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:47.945250 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:02:47.945361 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:47.945937 (+ 576us) server_negotiation.cc:299] Negotiation successful
0809 02:02:47.946091 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 02:02:47.947036 6509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:47.932827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:35875 (local address 127.1.192.193:38329)
0809 02:02:47.933324 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:47.933368 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 02:02:47.934689 (+ 1321us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:02:47.935257 (+ 568us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:02:47.935265 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:02:47.935641 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:02:47.936121 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:47.936134 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:47.937086 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:47.937089 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:47.938505 (+ 1416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:47.938516 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:47.939021 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:47.939029 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:47.939278 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:47.939932 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:02:47.939959 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:02:47.941865 (+ 1906us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:02:47.943991 (+ 2126us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:47.944001 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:47.944014 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:47.944318 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:47.944684 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:47.944687 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:47.944688 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:47.944832 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:47.945357 (+ 525us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:02:47.945362 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:02:47.945673 (+ 311us) client_negotiation.cc:769] Sending connection context
0809 02:02:47.945833 (+ 160us) client_negotiation.cc:240] Negotiation successful
0809 02:02:47.946107 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":130,"threads_started":1}
WARNING: no policy specified for kudu/127.1.192.194@KRBTEST.COM; defaulting to no policy
I20260809 02:02:47.948426 6507 heartbeater.cc:347] Connected to a master server at 127.1.192.254:35875
I20260809 02:02:47.948709 6507 heartbeater.cc:464] Registering TS with master...
I20260809 02:02:47.949178 6507 heartbeater.cc:511] Master 127.1.192.254:35875 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.192.194@KRBTEST.COM" created.
I20260809 02:02:47.950735 6297 ts_manager.cc:194] Registered new tserver with Master: fc036ebbd7964e0e93de22efe4ffc180 (127.1.192.193:39963)
I20260809 02:02:47.951920 6297 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.192.193@KRBTEST.COM'} at 127.1.192.193:38329
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.192.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.194.keytab.
Entry for principal kudu/127.1.192.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.194.keytab.
I20260809 02:02:47.969424 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786240941968412-1795-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786240941968412-1795-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786240941968412-1795-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786240941968412-1795-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.192.194:0
--local_ip_for_outbound_sockets=127.1.192.194
--webserver_interface=127.1.192.194
--webserver_port=0
--tserver_master_addrs=127.1.192.254:35875
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.194.keytab
--principal=kudu/127.1.192.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:44077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:02:48.082211 6512 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:02:48.082474 6512 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:02:48.082549 6512 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:02:48.086615 6512 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:02:48.086690 6512 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:02:48.086804 6512 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.192.194
I20260809 02:02:48.092605 6512 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:44077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786240941968412-1795-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786240941968412-1795-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.194.keytab
--principal=kudu/127.1.192.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.192.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786240941968412-1795-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.192.194
--webserver_port=0
--tserver_master_addrs=127.1.192.254:35875
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6512
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.192.194
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786240941968412-1795-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:02:48.093781 6512 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:02:48.094841 6512 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:02:48.104609 6521 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:02:48.104672 6518 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:02:48.104672 6519 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:02:48.104727 6512 server_base.cc:1034] running on GCE node
I20260809 02:02:48.105314 6512 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:02:48.106102 6512 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:02:48.107328 6512 hybrid_clock.cc:648] HybridClock initialized: now 1786240968107298 us; error 46 us; skew 500 ppm
Aug 09 02:02:48 dist-test-slave-ss4n krb5kdc[6248](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240968, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:02:48.110127 6512 init.cc:377] Logged in from keytab as kudu/127.1.192.194@KRBTEST.COM (short username kudu)
I20260809 02:02:48.110813 6512 fs_manager.cc:359] Metadata directory not provided
I20260809 02:02:48.110875 6512 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:02:48.111039 6512 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:02:48.113030 6512 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786240941968412-1795-0/minicluster-data/ts-1/data/instance:
uuid: "46639ece4aa74047af819f37ffb9a51b"
format_stamp: "Formatted at 2026-08-09 02:02:48 on dist-test-slave-ss4n"
server_key: "8f6190dde058b9f41ddcf1544356cc3c"
server_key_iv: "31160f8aa8ac11930e37340527cd5950"
server_key_version: "encryptionkey@0"
I20260809 02:02:48.113622 6512 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786240941968412-1795-0/minicluster-data/ts-1/wal/instance:
uuid: "46639ece4aa74047af819f37ffb9a51b"
format_stamp: "Formatted at 2026-08-09 02:02:48 on dist-test-slave-ss4n"
server_key: "8f6190dde058b9f41ddcf1544356cc3c"
server_key_iv: "31160f8aa8ac11930e37340527cd5950"
server_key_version: "encryptionkey@0"
I20260809 02:02:48.117230 6512 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260809 02:02:48.119606 6528 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:48.120785 6512 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 02:02:48.120878 6512 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786240941968412-1795-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786240941968412-1795-0/minicluster-data/ts-1/wal
uuid: "46639ece4aa74047af819f37ffb9a51b"
format_stamp: "Formatted at 2026-08-09 02:02:48 on dist-test-slave-ss4n"
server_key: "8f6190dde058b9f41ddcf1544356cc3c"
server_key_iv: "31160f8aa8ac11930e37340527cd5950"
server_key_version: "encryptionkey@0"
I20260809 02:02:48.120959 6512 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786240941968412-1795-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786240941968412-1795-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786240941968412-1795-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:02:48.139163 6512 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:02:48.142041 6512 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:02:48.142222 6512 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:02:48.142761 6512 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:02:48.143529 6512 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:02:48.143575 6512 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:48.143623 6512 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:02:48.143646 6512 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:48.152339 6512 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.194:46371
I20260809 02:02:48.152360 6655 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.194:46371 every 8 connection(s)
I20260809 02:02:48.153043 6512 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786240941968412-1795-0/minicluster-data/ts-1/data/info.pb
I20260809 02:02:48.156491 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 6512
I20260809 02:02:48.156594 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786240941968412-1795-0/minicluster-data/ts-1/wal/instance
I20260809 02:02:48.156912 1795 external_mini_cluster.cc:1428] Setting key a54bbaf7ca7293de37f6db7e697ce616
Aug 09 02:02:48 dist-test-slave-ss4n krb5kdc[6248](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240968, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.194@KRBTEST.COM for kudu/127.1.192.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.192.195@KRBTEST.COM; defaulting to no policy
I20260809 02:02:48.168488 6338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:48.154895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.194:57355 (local address 127.1.192.254:35875)
0809 02:02:48.155049 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:48.155054 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:48.155954 (+ 900us) server_negotiation.cc:407] Connection header received
0809 02:02:48.157671 (+ 1717us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:48.157675 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:48.157736 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:48.157844 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:48.159348 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:48.159876 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:48.161157 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:48.161410 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:48.161653 (+ 243us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:48.164616 (+ 2963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:02:48.164620 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:02:48.164625 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:02:48.164656 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:02:48.166273 (+ 1617us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:48.166846 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:48.166850 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:48.166852 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:48.166937 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:48.167293 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:48.167298 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:48.167300 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:48.167498 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:02:48.167607 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:48.168174 (+ 567us) server_negotiation.cc:299] Negotiation successful
0809 02:02:48.168307 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 02:02:48.169443 6658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:48.155342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:35875 (local address 127.1.192.194:57355)
0809 02:02:48.155795 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:48.155838 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 02:02:48.157233 (+ 1395us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:02:48.158052 (+ 819us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:02:48.158060 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:02:48.158656 (+ 596us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:02:48.159192 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:48.159207 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:48.160013 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:48.160016 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:48.161005 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:48.161019 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:48.161673 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:48.161679 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:48.161901 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:48.162623 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:02:48.162648 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:02:48.164436 (+ 1788us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:02:48.166399 (+ 1963us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:48.166405 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:48.166421 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:48.166734 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:48.167037 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:48.167041 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:48.167043 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:48.167196 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:48.167627 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:02:48.167634 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:02:48.167886 (+ 252us) client_negotiation.cc:769] Sending connection context
0809 02:02:48.168101 (+ 215us) client_negotiation.cc:240] Negotiation successful
0809 02:02:48.168446 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":122,"threads_started":1}
I20260809 02:02:48.170753 6656 heartbeater.cc:347] Connected to a master server at 127.1.192.254:35875
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.192.195@KRBTEST.COM" created.
I20260809 02:02:48.171059 6656 heartbeater.cc:464] Registering TS with master...
I20260809 02:02:48.171622 6656 heartbeater.cc:511] Master 127.1.192.254:35875 requested a full tablet report, sending...
I20260809 02:02:48.172784 6297 ts_manager.cc:194] Registered new tserver with Master: 46639ece4aa74047af819f37ffb9a51b (127.1.192.194:46371)
I20260809 02:02:48.173694 6297 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.192.194@KRBTEST.COM'} at 127.1.192.194:57355
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.192.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.195.keytab.
Entry for principal kudu/127.1.192.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.195.keytab.
I20260809 02:02:48.190412 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786240941968412-1795-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786240941968412-1795-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786240941968412-1795-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786240941968412-1795-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.192.195:0
--local_ip_for_outbound_sockets=127.1.192.195
--webserver_interface=127.1.192.195
--webserver_port=0
--tserver_master_addrs=127.1.192.254:35875
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.195.keytab
--principal=kudu/127.1.192.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:44077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:02:48.303771 6661 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:02:48.304046 6661 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:02:48.304095 6661 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:02:48.308430 6661 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:02:48.308501 6661 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:02:48.308566 6661 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.192.195
I20260809 02:02:48.314096 6661 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:44077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786240941968412-1795-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786240941968412-1795-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.195.keytab
--principal=kudu/127.1.192.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.192.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786240941968412-1795-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.192.195
--webserver_port=0
--tserver_master_addrs=127.1.192.254:35875
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6661
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.192.195
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786240941968412-1795-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:02:48.315213 6661 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:02:48.316274 6661 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:02:48.326355 6666 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:02:48.326431 6661 server_base.cc:1034] running on GCE node
W20260809 02:02:48.326344 6667 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:02:48.326350 6669 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:02:48.327051 6661 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:02:48.327894 6661 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:02:48.329077 6661 hybrid_clock.cc:648] HybridClock initialized: now 1786240968329038 us; error 52 us; skew 500 ppm
Aug 09 02:02:48 dist-test-slave-ss4n krb5kdc[6248](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240968, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:02:48.332090 6661 init.cc:377] Logged in from keytab as kudu/127.1.192.195@KRBTEST.COM (short username kudu)
I20260809 02:02:48.332871 6661 fs_manager.cc:359] Metadata directory not provided
I20260809 02:02:48.332971 6661 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:02:48.333227 6661 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:02:48.335374 6661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786240941968412-1795-0/minicluster-data/ts-2/data/instance:
uuid: "964bd9a9d5ff42278c1244d9a9f6debe"
format_stamp: "Formatted at 2026-08-09 02:02:48 on dist-test-slave-ss4n"
server_key: "d77dce79e2cf84e18218f99e0ff08b82"
server_key_iv: "f3b850f149dfddd4ebecaadfac677b29"
server_key_version: "encryptionkey@0"
I20260809 02:02:48.335956 6661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786240941968412-1795-0/minicluster-data/ts-2/wal/instance:
uuid: "964bd9a9d5ff42278c1244d9a9f6debe"
format_stamp: "Formatted at 2026-08-09 02:02:48 on dist-test-slave-ss4n"
server_key: "d77dce79e2cf84e18218f99e0ff08b82"
server_key_iv: "f3b850f149dfddd4ebecaadfac677b29"
server_key_version: "encryptionkey@0"
I20260809 02:02:48.339927 6661 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 02:02:48.342469 6675 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:48.343699 6661 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 02:02:48.343843 6661 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786240941968412-1795-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786240941968412-1795-0/minicluster-data/ts-2/wal
uuid: "964bd9a9d5ff42278c1244d9a9f6debe"
format_stamp: "Formatted at 2026-08-09 02:02:48 on dist-test-slave-ss4n"
server_key: "d77dce79e2cf84e18218f99e0ff08b82"
server_key_iv: "f3b850f149dfddd4ebecaadfac677b29"
server_key_version: "encryptionkey@0"
I20260809 02:02:48.343962 6661 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786240941968412-1795-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786240941968412-1795-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786240941968412-1795-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:02:48.355177 6661 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:02:48.358384 6661 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:02:48.358611 6661 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:02:48.359257 6661 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:02:48.360179 6661 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:02:48.360252 6661 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:48.360325 6661 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:02:48.360369 6661 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:48.370066 6661 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.195:41865
I20260809 02:02:48.370101 6799 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.195:41865 every 8 connection(s)
I20260809 02:02:48.370769 6661 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786240941968412-1795-0/minicluster-data/ts-2/data/info.pb
I20260809 02:02:48.377647 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 6661
I20260809 02:02:48.377815 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786240941968412-1795-0/minicluster-data/ts-2/wal/instance
I20260809 02:02:48.378103 1795 external_mini_cluster.cc:1428] Setting key fd57e453c8e5aecba832d3b425daa1a8
Aug 09 02:02:48 dist-test-slave-ss4n krb5kdc[6248](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240968, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.195@KRBTEST.COM for kudu/127.1.192.254@KRBTEST.COM
I20260809 02:02:48.386373 6338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:48.372822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.195:52393 (local address 127.1.192.254:35875)
0809 02:02:48.372982 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:48.372985 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:48.373932 (+ 947us) server_negotiation.cc:407] Connection header received
0809 02:02:48.375362 (+ 1430us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:48.375366 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:48.375428 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:48.375511 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:48.376723 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:48.377282 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:48.378739 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:48.379095 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:48.379218 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:48.382606 (+ 3388us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:02:48.382623 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:02:48.382627 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:02:48.382663 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:02:48.384123 (+ 1460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:48.384709 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:48.384712 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:48.384713 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:48.384789 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:48.385158 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:48.385161 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:48.385162 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:48.385345 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:02:48.385424 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:48.386137 (+ 713us) server_negotiation.cc:299] Negotiation successful
0809 02:02:48.386247 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 02:02:48.387394 6803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:48.373197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:35875 (local address 127.1.192.195:52393)
0809 02:02:48.373741 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:48.373787 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 02:02:48.375128 (+ 1341us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:02:48.375684 (+ 556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:02:48.375692 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:02:48.376073 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:02:48.376565 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:48.376580 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:48.377412 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:48.377415 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:48.378602 (+ 1187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:48.378615 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:48.379267 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:48.379270 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:48.379479 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:48.380285 (+ 806us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:02:48.380303 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:02:48.382380 (+ 2077us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:02:48.384234 (+ 1854us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:48.384239 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:48.384253 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:48.384602 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:48.384893 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:48.384895 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:48.384897 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:48.385045 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:48.385556 (+ 511us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:02:48.385564 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:02:48.385872 (+ 308us) client_negotiation.cc:769] Sending connection context
0809 02:02:48.386074 (+ 202us) client_negotiation.cc:240] Negotiation successful
0809 02:02:48.386373 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":119,"threads_started":1}
I20260809 02:02:48.388799 6800 heartbeater.cc:347] Connected to a master server at 127.1.192.254:35875
I20260809 02:02:48.389142 6800 heartbeater.cc:464] Registering TS with master...
I20260809 02:02:48.389740 6800 heartbeater.cc:511] Master 127.1.192.254:35875 requested a full tablet report, sending...
I20260809 02:02:48.390890 6297 ts_manager.cc:194] Registered new tserver with Master: 964bd9a9d5ff42278c1244d9a9f6debe (127.1.192.195:41865)
I20260809 02:02:48.391712 6297 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.192.195@KRBTEST.COM'} at 127.1.192.195:52393
I20260809 02:02:48.392567 1795 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 02:02:48.401686 6338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:48.394700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60740 (local address 127.1.192.254:35875)
0809 02:02:48.394870 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:48.394873 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:48.394945 (+ 72us) server_negotiation.cc:407] Connection header received
0809 02:02:48.395051 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:48.395053 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:48.395087 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:48.395156 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:48.395864 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:48.396374 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:48.397167 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:48.397373 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:48.397446 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:48.398908 (+ 1462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:02:48.398911 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:02:48.398913 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:02:48.398934 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:02:48.400187 (+ 1253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:48.400563 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:48.400565 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:48.400567 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:48.400637 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:48.400868 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:48.400870 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:48.400871 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:48.401072 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:02:48.401194 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:48.401370 (+ 176us) server_negotiation.cc:299] Negotiation successful
0809 02:02:48.401462 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260809 02:02:48.404657 6297 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60740:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 02:02:48.408222 6297 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:02:48.423272 6812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:48.416887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.193:39963 (local address 127.0.0.1:55066)
0809 02:02:48.417855 (+ 968us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:48.417910 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0809 02:02:48.418048 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:02:48.418905 (+ 857us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:02:48.418911 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:02:48.418936 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:02:48.419089 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:48.419096 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:48.420550 (+ 1454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:48.420553 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:48.421761 (+ 1208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:48.421770 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:48.422661 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:48.422664 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:48.422782 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:48.422823 (+ 41us) client_negotiation.cc:769] Sending connection context
0809 02:02:48.422972 (+ 149us) client_negotiation.cc:240] Negotiation successful
0809 02:02:48.423099 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":794,"mutex_wait_us":3,"thread_start_us":79,"threads_started":1}
I20260809 02:02:48.423444 6814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:48.417073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55066 (local address 127.1.192.193:39963)
0809 02:02:48.417414 (+ 341us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:48.417421 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:48.418207 (+ 786us) server_negotiation.cc:407] Connection header received
0809 02:02:48.418292 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:48.418300 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:48.418531 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:48.418688 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:02:48.419234 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:48.420410 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:48.421898 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:48.422523 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:48.422650 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:48.422805 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:48.423138 (+ 333us) server_negotiation.cc:299] Negotiation successful
0809 02:02:48.423270 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":222,"thread_start_us":129,"threads_started":1}
I20260809 02:02:48.423630 6818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:48.417969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35466 (local address 127.1.192.194:46371)
0809 02:02:48.418689 (+ 720us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:48.418701 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:48.418721 (+ 20us) server_negotiation.cc:407] Connection header received
0809 02:02:48.418796 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:48.418803 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:48.418967 (+ 164us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:48.419112 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:02:48.419390 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:48.420147 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:48.422047 (+ 1900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:48.422827 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:48.422932 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:48.423046 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:48.423348 (+ 302us) server_negotiation.cc:299] Negotiation successful
0809 02:02:48.423464 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":625,"thread_start_us":151,"threads_started":1}
I20260809 02:02:48.423280 6817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:48.417773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.194:46371 (local address 127.0.0.1:35466)
0809 02:02:48.418289 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:48.418301 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:02:48.418376 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:02:48.419097 (+ 721us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:02:48.419100 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:02:48.419116 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:02:48.419271 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:48.419282 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:48.420283 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:48.420288 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:48.421917 (+ 1629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:48.421925 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:48.422948 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:48.422954 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:48.423039 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:48.423060 (+ 21us) client_negotiation.cc:769] Sending connection context
0809 02:02:48.423127 (+ 67us) client_negotiation.cc:240] Negotiation successful
0809 02:02:48.423182 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":458,"mutex_wait_us":278,"spinlock_wait_cycles":11136,"thread_start_us":95,"threads_started":1}
I20260809 02:02:48.424011 6816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:48.417605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46852 (local address 127.1.192.195:41865)
0809 02:02:48.418145 (+ 540us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:48.418162 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:48.418434 (+ 272us) server_negotiation.cc:407] Connection header received
0809 02:02:48.418520 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:48.418528 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:48.418712 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:48.418855 (+ 143us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:02:48.419458 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:48.420530 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:48.421881 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:48.422715 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:48.422826 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:48.422971 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:48.423569 (+ 598us) server_negotiation.cc:299] Negotiation successful
0809 02:02:48.423786 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":427,"thread_start_us":186,"threads_started":1}
I20260809 02:02:48.424263 6815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:48.417107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.195:41865 (local address 127.0.0.1:46852)
0809 02:02:48.418208 (+ 1101us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:48.418223 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 02:02:48.418329 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:02:48.419128 (+ 799us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:02:48.419131 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:02:48.419154 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:02:48.419323 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:48.419353 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:48.420684 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:48.420689 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:48.421776 (+ 1087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:48.421787 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:48.423040 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:48.423044 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:48.423137 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:48.423163 (+ 26us) client_negotiation.cc:769] Sending connection context
0809 02:02:48.423235 (+ 72us) client_negotiation.cc:240] Negotiation successful
0809 02:02:48.423301 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1039,"mutex_wait_us":59,"thread_start_us":102,"threads_started":1}
I20260809 02:02:48.426647 6433 tablet_service.cc:1467] Processing CreateTablet for tablet a5e2bdf353bc4d7185da849408aba85f (DEFAULT_TABLE table=test-table [id=b3bc27f4ea3c483c9045efcd632ce5b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:02:48.426640 6582 tablet_service.cc:1467] Processing CreateTablet for tablet a5e2bdf353bc4d7185da849408aba85f (DEFAULT_TABLE table=test-table [id=b3bc27f4ea3c483c9045efcd632ce5b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:02:48.426868 6726 tablet_service.cc:1467] Processing CreateTablet for tablet a5e2bdf353bc4d7185da849408aba85f (DEFAULT_TABLE table=test-table [id=b3bc27f4ea3c483c9045efcd632ce5b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:02:48.427809 6433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a5e2bdf353bc4d7185da849408aba85f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:02:48.427809 6582 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a5e2bdf353bc4d7185da849408aba85f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:02:48.427840 6726 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a5e2bdf353bc4d7185da849408aba85f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:02:48.434788 6819 tablet_bootstrap.cc:492] T a5e2bdf353bc4d7185da849408aba85f P 964bd9a9d5ff42278c1244d9a9f6debe: Bootstrap starting.
I20260809 02:02:48.435632 6820 tablet_bootstrap.cc:492] T a5e2bdf353bc4d7185da849408aba85f P 46639ece4aa74047af819f37ffb9a51b: Bootstrap starting.
I20260809 02:02:48.436957 6819 tablet_bootstrap.cc:654] T a5e2bdf353bc4d7185da849408aba85f P 964bd9a9d5ff42278c1244d9a9f6debe: Neither blocks nor log segments found. Creating new log.
I20260809 02:02:48.437418 6820 tablet_bootstrap.cc:654] T a5e2bdf353bc4d7185da849408aba85f P 46639ece4aa74047af819f37ffb9a51b: Neither blocks nor log segments found. Creating new log.
I20260809 02:02:48.437920 6822 tablet_bootstrap.cc:492] T a5e2bdf353bc4d7185da849408aba85f P fc036ebbd7964e0e93de22efe4ffc180: Bootstrap starting.
I20260809 02:02:48.438287 6819 log.cc:824] T a5e2bdf353bc4d7185da849408aba85f P 964bd9a9d5ff42278c1244d9a9f6debe: Log is configured to *not* fsync() on all Append() calls
I20260809 02:02:48.438715 6820 log.cc:824] T a5e2bdf353bc4d7185da849408aba85f P 46639ece4aa74047af819f37ffb9a51b: Log is configured to *not* fsync() on all Append() calls
I20260809 02:02:48.440330 6822 tablet_bootstrap.cc:654] T a5e2bdf353bc4d7185da849408aba85f P fc036ebbd7964e0e93de22efe4ffc180: Neither blocks nor log segments found. Creating new log.
I20260809 02:02:48.440667 6819 tablet_bootstrap.cc:492] T a5e2bdf353bc4d7185da849408aba85f P 964bd9a9d5ff42278c1244d9a9f6debe: No bootstrap required, opened a new log
I20260809 02:02:48.440882 6819 ts_tablet_manager.cc:1397] T a5e2bdf353bc4d7185da849408aba85f P 964bd9a9d5ff42278c1244d9a9f6debe: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260809 02:02:48.440969 6820 tablet_bootstrap.cc:492] T a5e2bdf353bc4d7185da849408aba85f P 46639ece4aa74047af819f37ffb9a51b: No bootstrap required, opened a new log
I20260809 02:02:48.441175 6820 ts_tablet_manager.cc:1397] T a5e2bdf353bc4d7185da849408aba85f P 46639ece4aa74047af819f37ffb9a51b: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260809 02:02:48.441735 6822 log.cc:824] T a5e2bdf353bc4d7185da849408aba85f P fc036ebbd7964e0e93de22efe4ffc180: Log is configured to *not* fsync() on all Append() calls
I20260809 02:02:48.443851 6822 tablet_bootstrap.cc:492] T a5e2bdf353bc4d7185da849408aba85f P fc036ebbd7964e0e93de22efe4ffc180: No bootstrap required, opened a new log
I20260809 02:02:48.444043 6822 ts_tablet_manager.cc:1397] T a5e2bdf353bc4d7185da849408aba85f P fc036ebbd7964e0e93de22efe4ffc180: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.003s
I20260809 02:02:48.444375 6819 raft_consensus.cc:348] T a5e2bdf353bc4d7185da849408aba85f P 964bd9a9d5ff42278c1244d9a9f6debe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc036ebbd7964e0e93de22efe4ffc180" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 39963 } } peers { permanent_uuid: "964bd9a9d5ff42278c1244d9a9f6debe" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 41865 } } peers { permanent_uuid: "46639ece4aa74047af819f37ffb9a51b" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 46371 } }
I20260809 02:02:48.444643 6819 raft_consensus.cc:374] T a5e2bdf353bc4d7185da849408aba85f P 964bd9a9d5ff42278c1244d9a9f6debe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:02:48.444697 6819 raft_consensus.cc:729] T a5e2bdf353bc4d7185da849408aba85f P 964bd9a9d5ff42278c1244d9a9f6debe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 964bd9a9d5ff42278c1244d9a9f6debe, State: Initialized, Role: FOLLOWER
I20260809 02:02:48.444962 6820 raft_consensus.cc:348] T a5e2bdf353bc4d7185da849408aba85f P 46639ece4aa74047af819f37ffb9a51b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc036ebbd7964e0e93de22efe4ffc180" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 39963 } } peers { permanent_uuid: "964bd9a9d5ff42278c1244d9a9f6debe" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 41865 } } peers { permanent_uuid: "46639ece4aa74047af819f37ffb9a51b" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 46371 } }
I20260809 02:02:48.445236 6820 raft_consensus.cc:374] T a5e2bdf353bc4d7185da849408aba85f P 46639ece4aa74047af819f37ffb9a51b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:02:48.445293 6820 raft_consensus.cc:729] T a5e2bdf353bc4d7185da849408aba85f P 46639ece4aa74047af819f37ffb9a51b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46639ece4aa74047af819f37ffb9a51b, State: Initialized, Role: FOLLOWER
I20260809 02:02:48.445191 6819 consensus_queue.cc:260] T a5e2bdf353bc4d7185da849408aba85f P 964bd9a9d5ff42278c1244d9a9f6debe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc036ebbd7964e0e93de22efe4ffc180" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 39963 } } peers { permanent_uuid: "964bd9a9d5ff42278c1244d9a9f6debe" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 41865 } } peers { permanent_uuid: "46639ece4aa74047af819f37ffb9a51b" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 46371 } }
I20260809 02:02:48.445720 6820 consensus_queue.cc:260] T a5e2bdf353bc4d7185da849408aba85f P 46639ece4aa74047af819f37ffb9a51b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc036ebbd7964e0e93de22efe4ffc180" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 39963 } } peers { permanent_uuid: "964bd9a9d5ff42278c1244d9a9f6debe" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 41865 } } peers { permanent_uuid: "46639ece4aa74047af819f37ffb9a51b" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 46371 } }
I20260809 02:02:48.446308 6819 ts_tablet_manager.cc:1428] T a5e2bdf353bc4d7185da849408aba85f P 964bd9a9d5ff42278c1244d9a9f6debe: Time spent starting tablet: real 0.005s user 0.003s sys 0.003s
I20260809 02:02:48.446432 6656 heartbeater.cc:503] Master 127.1.192.254:35875 was elected leader, sending a full tablet report...
I20260809 02:02:48.447511 6800 heartbeater.cc:503] Master 127.1.192.254:35875 was elected leader, sending a full tablet report...
I20260809 02:02:48.448065 6822 raft_consensus.cc:348] T a5e2bdf353bc4d7185da849408aba85f P fc036ebbd7964e0e93de22efe4ffc180 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc036ebbd7964e0e93de22efe4ffc180" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 39963 } } peers { permanent_uuid: "964bd9a9d5ff42278c1244d9a9f6debe" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 41865 } } peers { permanent_uuid: "46639ece4aa74047af819f37ffb9a51b" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 46371 } }
I20260809 02:02:48.448340 6822 raft_consensus.cc:374] T a5e2bdf353bc4d7185da849408aba85f P fc036ebbd7964e0e93de22efe4ffc180 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:02:48.448397 6822 raft_consensus.cc:729] T a5e2bdf353bc4d7185da849408aba85f P fc036ebbd7964e0e93de22efe4ffc180 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc036ebbd7964e0e93de22efe4ffc180, State: Initialized, Role: FOLLOWER
I20260809 02:02:48.448838 6822 consensus_queue.cc:260] T a5e2bdf353bc4d7185da849408aba85f P fc036ebbd7964e0e93de22efe4ffc180 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc036ebbd7964e0e93de22efe4ffc180" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 39963 } } peers { permanent_uuid: "964bd9a9d5ff42278c1244d9a9f6debe" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 41865 } } peers { permanent_uuid: "46639ece4aa74047af819f37ffb9a51b" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 46371 } }
I20260809 02:02:48.448957 6820 ts_tablet_manager.cc:1428] T a5e2bdf353bc4d7185da849408aba85f P 46639ece4aa74047af819f37ffb9a51b: Time spent starting tablet: real 0.008s user 0.003s sys 0.003s
I20260809 02:02:48.449609 6507 heartbeater.cc:503] Master 127.1.192.254:35875 was elected leader, sending a full tablet report...
I20260809 02:02:48.449817 6822 ts_tablet_manager.cc:1428] T a5e2bdf353bc4d7185da849408aba85f P fc036ebbd7964e0e93de22efe4ffc180: Time spent starting tablet: real 0.006s user 0.004s sys 0.003s
W20260809 02:02:48.621958 6801 tablet.cc:2369] T a5e2bdf353bc4d7185da849408aba85f P 964bd9a9d5ff42278c1244d9a9f6debe: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 02:02:48.654322 6657 tablet.cc:2369] T a5e2bdf353bc4d7185da849408aba85f P 46639ece4aa74047af819f37ffb9a51b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 02:02:48.682050 6508 tablet.cc:2369] T a5e2bdf353bc4d7185da849408aba85f P fc036ebbd7964e0e93de22efe4ffc180: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 02:02:48.734803 6830 raft_consensus.cc:482] T a5e2bdf353bc4d7185da849408aba85f P fc036ebbd7964e0e93de22efe4ffc180 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:02:48.735047 6830 raft_consensus.cc:504] T a5e2bdf353bc4d7185da849408aba85f P fc036ebbd7964e0e93de22efe4ffc180 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc036ebbd7964e0e93de22efe4ffc180" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 39963 } } peers { permanent_uuid: "964bd9a9d5ff42278c1244d9a9f6debe" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 41865 } } peers { permanent_uuid: "46639ece4aa74047af819f37ffb9a51b" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 46371 } }
I20260809 02:02:48.736231 6830 leader_election.cc:288] T a5e2bdf353bc4d7185da849408aba85f P fc036ebbd7964e0e93de22efe4ffc180 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 964bd9a9d5ff42278c1244d9a9f6debe (127.1.192.195:41865), 46639ece4aa74047af819f37ffb9a51b (127.1.192.194:46371)
I20260809 02:02:48.742080 6816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:48.736602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.193:39803 (local address 127.1.192.195:41865)
0809 02:02:48.736759 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:48.736764 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:48.736992 (+ 228us) server_negotiation.cc:407] Connection header received
0809 02:02:48.737152 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:48.737156 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:48.737206 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:48.737295 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:02:48.737775 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:48.738561 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:48.740535 (+ 1974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:48.741225 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:48.741379 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:48.741437 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:48.741835 (+ 398us) server_negotiation.cc:299] Negotiation successful
0809 02:02:48.741920 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 02:02:48.742080 6818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:48.736652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.193:54387 (local address 127.1.192.194:46371)
0809 02:02:48.736873 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:48.736876 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:48.736990 (+ 114us) server_negotiation.cc:407] Connection header received
0809 02:02:48.737152 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:48.737156 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:48.737206 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:48.737295 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:02:48.737775 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:48.738570 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:48.740510 (+ 1940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:48.741233 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:48.741379 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:48.741437 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:48.741835 (+ 398us) server_negotiation.cc:299] Negotiation successful
0809 02:02:48.741920 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136}
I20260809 02:02:48.742123 6832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:48.736489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.194:46371 (local address 127.1.192.193:54387)
0809 02:02:48.736871 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:48.736887 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 02:02:48.737036 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:02:48.737327 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:02:48.737330 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:02:48.737358 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:02:48.737627 (+ 269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:48.737632 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:48.738694 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:48.738698 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:48.740330 (+ 1632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:48.740343 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:48.741380 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:48.741384 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:48.741610 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:48.741711 (+ 101us) client_negotiation.cc:769] Sending connection context
0809 02:02:48.741790 (+ 79us) client_negotiation.cc:240] Negotiation successful
0809 02:02:48.741933 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"mutex_wait_us":329,"spinlock_wait_cycles":20736,"thread_start_us":139,"threads_started":1}
I20260809 02:02:48.742127 6831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:48.736493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.195:41865 (local address 127.1.192.193:39803)
0809 02:02:48.736871 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:48.736887 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 02:02:48.737036 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:02:48.737327 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:02:48.737330 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:02:48.737358 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:02:48.737623 (+ 265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:48.737631 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:48.738694 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:48.738698 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:48.740330 (+ 1632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:48.740343 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:48.741380 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:48.741384 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:48.741582 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:48.741703 (+ 121us) client_negotiation.cc:769] Sending connection context
0809 02:02:48.741790 (+ 87us) client_negotiation.cc:240] Negotiation successful
0809 02:02:48.741933 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":325,"mutex_wait_us":362,"spinlock_wait_cycles":23168,"thread_start_us":114,"threads_started":1}
I20260809 02:02:48.742894 6607 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5e2bdf353bc4d7185da849408aba85f" candidate_uuid: "fc036ebbd7964e0e93de22efe4ffc180" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46639ece4aa74047af819f37ffb9a51b" is_pre_election: true
I20260809 02:02:48.743139 6749 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5e2bdf353bc4d7185da849408aba85f" candidate_uuid: "fc036ebbd7964e0e93de22efe4ffc180" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "964bd9a9d5ff42278c1244d9a9f6debe" is_pre_election: true
I20260809 02:02:48.743228 6607 raft_consensus.cc:2454] T a5e2bdf353bc4d7185da849408aba85f P 46639ece4aa74047af819f37ffb9a51b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc036ebbd7964e0e93de22efe4ffc180 in term 0.
I20260809 02:02:48.743428 6749 raft_consensus.cc:2454] T a5e2bdf353bc4d7185da849408aba85f P 964bd9a9d5ff42278c1244d9a9f6debe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc036ebbd7964e0e93de22efe4ffc180 in term 0.
I20260809 02:02:48.743764 6385 leader_election.cc:302] T a5e2bdf353bc4d7185da849408aba85f P fc036ebbd7964e0e93de22efe4ffc180 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46639ece4aa74047af819f37ffb9a51b, fc036ebbd7964e0e93de22efe4ffc180; no voters:
I20260809 02:02:48.744071 6830 raft_consensus.cc:2781] T a5e2bdf353bc4d7185da849408aba85f P fc036ebbd7964e0e93de22efe4ffc180 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:02:48.744161 6830 raft_consensus.cc:482] T a5e2bdf353bc4d7185da849408aba85f P fc036ebbd7964e0e93de22efe4ffc180 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:02:48.744207 6830 raft_consensus.cc:3037] T a5e2bdf353bc4d7185da849408aba85f P fc036ebbd7964e0e93de22efe4ffc180 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:02:48.745234 6830 raft_consensus.cc:504] T a5e2bdf353bc4d7185da849408aba85f P fc036ebbd7964e0e93de22efe4ffc180 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc036ebbd7964e0e93de22efe4ffc180" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 39963 } } peers { permanent_uuid: "964bd9a9d5ff42278c1244d9a9f6debe" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 41865 } } peers { permanent_uuid: "46639ece4aa74047af819f37ffb9a51b" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 46371 } }
I20260809 02:02:48.745646 6830 leader_election.cc:288] T a5e2bdf353bc4d7185da849408aba85f P fc036ebbd7964e0e93de22efe4ffc180 [CANDIDATE]: Term 1 election: Requested vote from peers 964bd9a9d5ff42278c1244d9a9f6debe (127.1.192.195:41865), 46639ece4aa74047af819f37ffb9a51b (127.1.192.194:46371)
I20260809 02:02:48.746006 6607 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5e2bdf353bc4d7185da849408aba85f" candidate_uuid: "fc036ebbd7964e0e93de22efe4ffc180" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46639ece4aa74047af819f37ffb9a51b"
I20260809 02:02:48.745993 6749 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a5e2bdf353bc4d7185da849408aba85f" candidate_uuid: "fc036ebbd7964e0e93de22efe4ffc180" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "964bd9a9d5ff42278c1244d9a9f6debe"
I20260809 02:02:48.746142 6749 raft_consensus.cc:3037] T a5e2bdf353bc4d7185da849408aba85f P 964bd9a9d5ff42278c1244d9a9f6debe [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:02:48.746151 6607 raft_consensus.cc:3037] T a5e2bdf353bc4d7185da849408aba85f P 46639ece4aa74047af819f37ffb9a51b [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:02:48.747462 6749 raft_consensus.cc:2454] T a5e2bdf353bc4d7185da849408aba85f P 964bd9a9d5ff42278c1244d9a9f6debe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc036ebbd7964e0e93de22efe4ffc180 in term 1.
I20260809 02:02:48.747469 6607 raft_consensus.cc:2454] T a5e2bdf353bc4d7185da849408aba85f P 46639ece4aa74047af819f37ffb9a51b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc036ebbd7964e0e93de22efe4ffc180 in term 1.
I20260809 02:02:48.747781 6385 leader_election.cc:302] T a5e2bdf353bc4d7185da849408aba85f P fc036ebbd7964e0e93de22efe4ffc180 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 46639ece4aa74047af819f37ffb9a51b, 964bd9a9d5ff42278c1244d9a9f6debe, fc036ebbd7964e0e93de22efe4ffc180; no voters:
I20260809 02:02:48.747931 6830 raft_consensus.cc:2781] T a5e2bdf353bc4d7185da849408aba85f P fc036ebbd7964e0e93de22efe4ffc180 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:02:48.748162 6830 raft_consensus.cc:686] T a5e2bdf353bc4d7185da849408aba85f P fc036ebbd7964e0e93de22efe4ffc180 [term 1 LEADER]: Becoming Leader. State: Replica: fc036ebbd7964e0e93de22efe4ffc180, State: Running, Role: LEADER
I20260809 02:02:48.748515 6830 consensus_queue.cc:237] T a5e2bdf353bc4d7185da849408aba85f P fc036ebbd7964e0e93de22efe4ffc180 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc036ebbd7964e0e93de22efe4ffc180" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 39963 } } peers { permanent_uuid: "964bd9a9d5ff42278c1244d9a9f6debe" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 41865 } } peers { permanent_uuid: "46639ece4aa74047af819f37ffb9a51b" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 46371 } }
I20260809 02:02:48.751971 6296 catalog_manager.cc:5495] T a5e2bdf353bc4d7185da849408aba85f P fc036ebbd7964e0e93de22efe4ffc180 reported cstate change: term changed from 0 to 1, leader changed from <none> to fc036ebbd7964e0e93de22efe4ffc180 (127.1.192.193). New cstate: current_term: 1 leader_uuid: "fc036ebbd7964e0e93de22efe4ffc180" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc036ebbd7964e0e93de22efe4ffc180" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 39963 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "964bd9a9d5ff42278c1244d9a9f6debe" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 41865 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46639ece4aa74047af819f37ffb9a51b" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 46371 } health_report { overall_health: UNKNOWN } } }
I20260809 02:02:48.788697 6814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:48.785043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55070 (local address 127.1.192.193:39963)
0809 02:02:48.785200 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:48.785204 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:48.785218 (+ 14us) server_negotiation.cc:407] Connection header received
0809 02:02:48.785353 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:48.785356 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:48.785410 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:48.785559 (+ 149us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:02:48.785794 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:48.786299 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:48.787133 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:48.787337 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:48.787409 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:48.787736 (+ 327us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:02:48.788211 (+ 475us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:02:48.788356 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:48.788397 (+ 41us) server_negotiation.cc:299] Negotiation successful
0809 02:02:48.788456 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 02:02:48.798638 6607 raft_consensus.cc:1261] T a5e2bdf353bc4d7185da849408aba85f P 46639ece4aa74047af819f37ffb9a51b [term 1 FOLLOWER]: Refusing update from remote peer fc036ebbd7964e0e93de22efe4ffc180: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:02:48.799180 6749 raft_consensus.cc:1261] T a5e2bdf353bc4d7185da849408aba85f P 964bd9a9d5ff42278c1244d9a9f6debe [term 1 FOLLOWER]: Refusing update from remote peer fc036ebbd7964e0e93de22efe4ffc180: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:02:48.799460 6830 consensus_queue.cc:1035] T a5e2bdf353bc4d7185da849408aba85f P fc036ebbd7964e0e93de22efe4ffc180 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46639ece4aa74047af819f37ffb9a51b" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 46371 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:02:48.799916 6833 consensus_queue.cc:1035] T a5e2bdf353bc4d7185da849408aba85f P fc036ebbd7964e0e93de22efe4ffc180 [LEADER]: Connected to new peer: Peer: permanent_uuid: "964bd9a9d5ff42278c1244d9a9f6debe" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 41865 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:02:48.810360 6838 mvcc.cc:205] Tried to move back new op lower bound from 7316443008177729536 to 7316443007996620800. Current Snapshot: MvccSnapshot[applied={T|T < 7316443008177729536}]
I20260809 02:02:48.815541 6296 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60740:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:02:48.815704 6296 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60740:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:02:48.818107 6296 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8e9ac229743b44908e5b8a96323f8ca1: Sending DeleteTablet for 3 replicas of tablet a5e2bdf353bc4d7185da849408aba85f
I20260809 02:02:48.819003 6433 tablet_service.cc:1514] Processing DeleteTablet for tablet a5e2bdf353bc4d7185da849408aba85f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:02:48 UTC) from {username='kudu', principal='kudu/127.1.192.254@KRBTEST.COM'} at 127.0.0.1:55066
I20260809 02:02:48.818990 6582 tablet_service.cc:1514] Processing DeleteTablet for tablet a5e2bdf353bc4d7185da849408aba85f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:02:48 UTC) from {username='kudu', principal='kudu/127.1.192.254@KRBTEST.COM'} at 127.0.0.1:35466
I20260809 02:02:48.819504 6849 tablet_replica.cc:330] T a5e2bdf353bc4d7185da849408aba85f P fc036ebbd7964e0e93de22efe4ffc180: stopping tablet replica
I20260809 02:02:48.819813 6726 tablet_service.cc:1514] Processing DeleteTablet for tablet a5e2bdf353bc4d7185da849408aba85f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:02:48 UTC) from {username='kudu', principal='kudu/127.1.192.254@KRBTEST.COM'} at 127.0.0.1:46852
I20260809 02:02:48.819902 6849 raft_consensus.cc:2229] T a5e2bdf353bc4d7185da849408aba85f P fc036ebbd7964e0e93de22efe4ffc180 [term 1 LEADER]: Raft consensus shutting down.
I20260809 02:02:48.820282 6850 tablet_replica.cc:330] T a5e2bdf353bc4d7185da849408aba85f P 964bd9a9d5ff42278c1244d9a9f6debe: stopping tablet replica
I20260809 02:02:48.820399 6849 raft_consensus.cc:2258] T a5e2bdf353bc4d7185da849408aba85f P fc036ebbd7964e0e93de22efe4ffc180 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:02:48.820429 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 6366
I20260809 02:02:48.821120 6847 tablet_replica.cc:330] T a5e2bdf353bc4d7185da849408aba85f P 46639ece4aa74047af819f37ffb9a51b: stopping tablet replica
I20260809 02:02:48.821403 6847 raft_consensus.cc:2229] T a5e2bdf353bc4d7185da849408aba85f P 46639ece4aa74047af819f37ffb9a51b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:02:48.821740 6847 raft_consensus.cc:2258] T a5e2bdf353bc4d7185da849408aba85f P 46639ece4aa74047af819f37ffb9a51b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:02:48.823191 6847 ts_tablet_manager.cc:1905] T a5e2bdf353bc4d7185da849408aba85f P 46639ece4aa74047af819f37ffb9a51b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:02:48.824429 6850 raft_consensus.cc:2229] T a5e2bdf353bc4d7185da849408aba85f P 964bd9a9d5ff42278c1244d9a9f6debe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:02:48.824805 6850 raft_consensus.cc:2258] T a5e2bdf353bc4d7185da849408aba85f P 964bd9a9d5ff42278c1244d9a9f6debe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:02:48.826014 6847 ts_tablet_manager.cc:1918] T a5e2bdf353bc4d7185da849408aba85f P 46639ece4aa74047af819f37ffb9a51b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:02:48.826143 6847 log.cc:1196] T a5e2bdf353bc4d7185da849408aba85f P 46639ece4aa74047af819f37ffb9a51b: Deleting WAL directory at /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786240941968412-1795-0/minicluster-data/ts-1/wal/wals/a5e2bdf353bc4d7185da849408aba85f
I20260809 02:02:48.826507 6847 ts_tablet_manager.cc:1939] T a5e2bdf353bc4d7185da849408aba85f P 46639ece4aa74047af819f37ffb9a51b: Deleting consensus metadata
I20260809 02:02:48.826503 6850 ts_tablet_manager.cc:1905] T a5e2bdf353bc4d7185da849408aba85f P 964bd9a9d5ff42278c1244d9a9f6debe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:02:48.827901 6282 catalog_manager.cc:4841] TS 46639ece4aa74047af819f37ffb9a51b (127.1.192.194:46371): tablet a5e2bdf353bc4d7185da849408aba85f (table test-table [id=b3bc27f4ea3c483c9045efcd632ce5b8]) successfully deleted
I20260809 02:02:48.829576 6850 ts_tablet_manager.cc:1918] T a5e2bdf353bc4d7185da849408aba85f P 964bd9a9d5ff42278c1244d9a9f6debe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:02:48.829685 6850 log.cc:1196] T a5e2bdf353bc4d7185da849408aba85f P 964bd9a9d5ff42278c1244d9a9f6debe: Deleting WAL directory at /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786240941968412-1795-0/minicluster-data/ts-2/wal/wals/a5e2bdf353bc4d7185da849408aba85f
I20260809 02:02:48.830019 6850 ts_tablet_manager.cc:1939] T a5e2bdf353bc4d7185da849408aba85f P 964bd9a9d5ff42278c1244d9a9f6debe: Deleting consensus metadata
I20260809 02:02:48.831103 6283 catalog_manager.cc:4841] TS 964bd9a9d5ff42278c1244d9a9f6debe (127.1.192.195:41865): tablet a5e2bdf353bc4d7185da849408aba85f (table test-table [id=b3bc27f4ea3c483c9045efcd632ce5b8]) successfully deleted
W20260809 02:02:48.833001 6282 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.192.193:39963 (error 108)
I20260809 02:02:48.833205 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 6512
I20260809 02:02:48.835775 6815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:48.835411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.193:39963 (local address 127.0.0.1:55082)
0809 02:02:48.835630 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:48.835698 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.193:39963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260809 02:02:48.835945 6282 catalog_manager.cc:4584] TS fc036ebbd7964e0e93de22efe4ffc180 (127.1.192.193:39963): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a5e2bdf353bc4d7185da849408aba85f: Network error: Client connection negotiation failed: client connection to 127.1.192.193:39963: connect: Connection refused (error 111)
I20260809 02:02:48.840583 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 6661
I20260809 02:02:48.848013 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 6264
2026-08-09T02:02:48Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (3465 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-09T02:02:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:02:48Z Disabled control of system clock
I20260809 02:02:48.881414 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786240941968412-1795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786240941968412-1795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786240941968412-1795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.192.254:41123
--webserver_interface=127.1.192.254
--webserver_port=0
--builtin_ntp_servers=127.1.192.212:40641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.192.254:41123
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 02:02:49.007405 6856 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:02:49.007855 6856 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:02:49.007937 6856 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:02:49.012116 6856 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:02:49.012184 6856 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:02:49.012202 6856 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:02:49.012219 6856 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:02:49.012235 6856 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:02:49.017274 6856 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:40641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786240941968412-1795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786240941968412-1795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.192.254:41123
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.192.254:41123
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.192.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6856
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786240941968412-1795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:02:49.018600 6856 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:02:49.020004 6856 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:02:49.028968 6862 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:02:49.029012 6861 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:02:49.029032 6865 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:02:49.029343 6856 server_base.cc:1034] running on GCE node
I20260809 02:02:49.029954 6856 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:02:49.031025 6856 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:02:49.032217 6856 hybrid_clock.cc:648] HybridClock initialized: now 1786240969032187 us; error 47 us; skew 500 ppm
I20260809 02:02:49.034211 6856 webserver.cc:460] Webserver started at http://127.1.192.254:41647/ using document root <none> and password file <none>
I20260809 02:02:49.034821 6856 fs_manager.cc:359] Metadata directory not provided
I20260809 02:02:49.034886 6856 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:02:49.035135 6856 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:02:49.037271 6856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786240941968412-1795-0/minicluster-data/master-0/data/instance:
uuid: "169f595f250a404883a589b70d084275"
format_stamp: "Formatted at 2026-08-09 02:02:49 on dist-test-slave-ss4n"
server_key: "8360d24e49cfb7ffc9fd9b96433085ea"
server_key_iv: "a33955490b00652144a22149126519a0"
server_key_version: "encryptionkey@0"
I20260809 02:02:49.037917 6856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786240941968412-1795-0/minicluster-data/master-0/wal/instance:
uuid: "169f595f250a404883a589b70d084275"
format_stamp: "Formatted at 2026-08-09 02:02:49 on dist-test-slave-ss4n"
server_key: "8360d24e49cfb7ffc9fd9b96433085ea"
server_key_iv: "a33955490b00652144a22149126519a0"
server_key_version: "encryptionkey@0"
I20260809 02:02:49.041909 6856 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260809 02:02:49.044427 6871 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:49.045884 6856 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 02:02:49.046015 6856 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786240941968412-1795-0/minicluster-data/master-0/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786240941968412-1795-0/minicluster-data/master-0/wal
uuid: "169f595f250a404883a589b70d084275"
format_stamp: "Formatted at 2026-08-09 02:02:49 on dist-test-slave-ss4n"
server_key: "8360d24e49cfb7ffc9fd9b96433085ea"
server_key_iv: "a33955490b00652144a22149126519a0"
server_key_version: "encryptionkey@0"
I20260809 02:02:49.046136 6856 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786240941968412-1795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786240941968412-1795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786240941968412-1795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:02:49.050807 6856 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:02:49.051468 6856 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:02:49.051669 6856 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:02:49.059031 6856 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.254:41123
I20260809 02:02:49.059058 6928 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.254:41123 every 8 connection(s)
I20260809 02:02:49.059985 6856 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
I20260809 02:02:49.063701 6929 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:02:49.067638 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 6856
I20260809 02:02:49.067775 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786240941968412-1795-0/minicluster-data/master-0/wal/instance
I20260809 02:02:49.068096 1795 external_mini_cluster.cc:1428] Setting key a94af86463e59dd5e3d7b1bc691aafc0
I20260809 02:02:49.070475 6929 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 169f595f250a404883a589b70d084275: Bootstrap starting.
I20260809 02:02:49.072937 6929 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 169f595f250a404883a589b70d084275: Neither blocks nor log segments found. Creating new log.
I20260809 02:02:49.073885 6929 log.cc:824] T 00000000000000000000000000000000 P 169f595f250a404883a589b70d084275: Log is configured to *not* fsync() on all Append() calls
I20260809 02:02:49.074357 6933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:49.069848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35706 (local address 127.1.192.254:41123)
0809 02:02:49.070583 (+ 735us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:49.070607 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:49.070655 (+ 48us) server_negotiation.cc:407] Connection header received
0809 02:02:49.071314 (+ 659us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:49.071331 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:49.071744 (+ 413us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:49.072087 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:49.072948 (+ 861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:02:49.072953 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:02:49.072965 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:02:49.072976 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:02:49.073066 (+ 90us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:02:49.073098 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:02:49.073221 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:49.073322 (+ 101us) server_negotiation.cc:299] Negotiation successful
0809 02:02:49.073601 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":171,"threads_started":1}
I20260809 02:02:49.075894 6929 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 169f595f250a404883a589b70d084275: No bootstrap required, opened a new log
I20260809 02:02:49.079418 6929 raft_consensus.cc:348] T 00000000000000000000000000000000 P 169f595f250a404883a589b70d084275 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "169f595f250a404883a589b70d084275" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 41123 } }
I20260809 02:02:49.079635 6929 raft_consensus.cc:374] T 00000000000000000000000000000000 P 169f595f250a404883a589b70d084275 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:02:49.079697 6929 raft_consensus.cc:729] T 00000000000000000000000000000000 P 169f595f250a404883a589b70d084275 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 169f595f250a404883a589b70d084275, State: Initialized, Role: FOLLOWER
I20260809 02:02:49.080090 6929 consensus_queue.cc:260] T 00000000000000000000000000000000 P 169f595f250a404883a589b70d084275 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "169f595f250a404883a589b70d084275" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 41123 } }
I20260809 02:02:49.080232 6929 raft_consensus.cc:388] T 00000000000000000000000000000000 P 169f595f250a404883a589b70d084275 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:02:49.080296 6929 raft_consensus.cc:482] T 00000000000000000000000000000000 P 169f595f250a404883a589b70d084275 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:02:49.080433 6929 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 169f595f250a404883a589b70d084275 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:02:49.081391 6929 raft_consensus.cc:504] T 00000000000000000000000000000000 P 169f595f250a404883a589b70d084275 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "169f595f250a404883a589b70d084275" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 41123 } }
I20260809 02:02:49.081761 6929 leader_election.cc:302] T 00000000000000000000000000000000 P 169f595f250a404883a589b70d084275 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 169f595f250a404883a589b70d084275; no voters:
I20260809 02:02:49.082044 6929 leader_election.cc:288] T 00000000000000000000000000000000 P 169f595f250a404883a589b70d084275 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:02:49.082201 6935 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 169f595f250a404883a589b70d084275 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:02:49.082455 6935 raft_consensus.cc:686] T 00000000000000000000000000000000 P 169f595f250a404883a589b70d084275 [term 1 LEADER]: Becoming Leader. State: Replica: 169f595f250a404883a589b70d084275, State: Running, Role: LEADER
I20260809 02:02:49.082762 6935 consensus_queue.cc:237] T 00000000000000000000000000000000 P 169f595f250a404883a589b70d084275 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "169f595f250a404883a589b70d084275" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 41123 } }
I20260809 02:02:49.083298 6929 sys_catalog.cc:564] T 00000000000000000000000000000000 P 169f595f250a404883a589b70d084275 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:02:49.084395 6937 sys_catalog.cc:455] T 00000000000000000000000000000000 P 169f595f250a404883a589b70d084275 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 169f595f250a404883a589b70d084275. Latest consensus state: current_term: 1 leader_uuid: "169f595f250a404883a589b70d084275" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "169f595f250a404883a589b70d084275" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 41123 } } }
I20260809 02:02:49.084539 6937 sys_catalog.cc:458] T 00000000000000000000000000000000 P 169f595f250a404883a589b70d084275 [sys.catalog]: This master's current role is: LEADER
I20260809 02:02:49.084827 6936 sys_catalog.cc:455] T 00000000000000000000000000000000 P 169f595f250a404883a589b70d084275 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "169f595f250a404883a589b70d084275" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "169f595f250a404883a589b70d084275" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 41123 } } }
I20260809 02:02:49.084928 6936 sys_catalog.cc:458] T 00000000000000000000000000000000 P 169f595f250a404883a589b70d084275 [sys.catalog]: This master's current role is: LEADER
I20260809 02:02:49.085008 6945 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:02:49.088127 6945 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:02:49.093899 6945 catalog_manager.cc:1347] Generated new cluster ID: 5809b4c903a540f3bcd469dd748d01e5
I20260809 02:02:49.093972 6945 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:02:49.130156 6945 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:02:49.130506 6945 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:02:49.136727 6945 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 169f595f250a404883a589b70d084275: Generated new TSK 0
I20260809 02:02:49.137589 6945 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:02:49.154229 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786240941968412-1795-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786240941968412-1795-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786240941968412-1795-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786240941968412-1795-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.192.193:0
--local_ip_for_outbound_sockets=127.1.192.193
--webserver_interface=127.1.192.193
--webserver_port=0
--tserver_master_addrs=127.1.192.254:41123
--builtin_ntp_servers=127.1.192.212:40641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 02:02:49.267597 6956 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:02:49.267880 6956 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:02:49.267922 6956 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:02:49.271863 6956 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:02:49.271919 6956 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:02:49.272032 6956 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.192.193
I20260809 02:02:49.276793 6956 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:40641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786240941968412-1795-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786240941968412-1795-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.192.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786240941968412-1795-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.192.193
--webserver_port=0
--tserver_master_addrs=127.1.192.254:41123
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6956
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.192.193
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786240941968412-1795-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:02:49.277882 6956 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:02:49.279245 6956 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:02:49.289021 6965 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:02:49.289057 6962 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:02:49.289026 6963 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:02:49.289089 6956 server_base.cc:1034] running on GCE node
I20260809 02:02:49.289722 6956 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:02:49.290360 6956 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:02:49.291554 6956 hybrid_clock.cc:648] HybridClock initialized: now 1786240969291531 us; error 34 us; skew 500 ppm
I20260809 02:02:49.293421 6956 webserver.cc:460] Webserver started at http://127.1.192.193:39069/ using document root <none> and password file <none>
I20260809 02:02:49.294077 6956 fs_manager.cc:359] Metadata directory not provided
I20260809 02:02:49.294153 6956 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:02:49.294421 6956 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:02:49.296597 6956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786240941968412-1795-0/minicluster-data/ts-0/data/instance:
uuid: "f9e1088d93c24328aa5b872dd8764046"
format_stamp: "Formatted at 2026-08-09 02:02:49 on dist-test-slave-ss4n"
server_key: "1791851483138305e8523af1b0ef5172"
server_key_iv: "63d60a871e618da9482624d40db22a94"
server_key_version: "encryptionkey@0"
I20260809 02:02:49.297175 6956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786240941968412-1795-0/minicluster-data/ts-0/wal/instance:
uuid: "f9e1088d93c24328aa5b872dd8764046"
format_stamp: "Formatted at 2026-08-09 02:02:49 on dist-test-slave-ss4n"
server_key: "1791851483138305e8523af1b0ef5172"
server_key_iv: "63d60a871e618da9482624d40db22a94"
server_key_version: "encryptionkey@0"
I20260809 02:02:49.301156 6956 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260809 02:02:49.303812 6971 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:49.305377 6956 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 02:02:49.305539 6956 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786240941968412-1795-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786240941968412-1795-0/minicluster-data/ts-0/wal
uuid: "f9e1088d93c24328aa5b872dd8764046"
format_stamp: "Formatted at 2026-08-09 02:02:49 on dist-test-slave-ss4n"
server_key: "1791851483138305e8523af1b0ef5172"
server_key_iv: "63d60a871e618da9482624d40db22a94"
server_key_version: "encryptionkey@0"
I20260809 02:02:49.305686 6956 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786240941968412-1795-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786240941968412-1795-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786240941968412-1795-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:02:49.310297 6956 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:02:49.310921 6956 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:02:49.311112 6956 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:02:49.311699 6956 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:02:49.312616 6956 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:02:49.312666 6956 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:49.312734 6956 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:02:49.312765 6956 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:49.321179 6956 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.193:39813
I20260809 02:02:49.321267 7090 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.193:39813 every 8 connection(s)
I20260809 02:02:49.322197 6956 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786240941968412-1795-0/minicluster-data/ts-0/data/info.pb
I20260809 02:02:49.328640 6933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:49.324063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.193:58643 (local address 127.1.192.254:41123)
0809 02:02:49.324212 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:49.324216 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:49.325310 (+ 1094us) server_negotiation.cc:407] Connection header received
0809 02:02:49.326843 (+ 1533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:49.326847 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:49.326895 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:49.326997 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:49.328160 (+ 1163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:02:49.328164 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:02:49.328166 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:02:49.328180 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:02:49.328209 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:02:49.328218 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:02:49.328321 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:49.328455 (+ 134us) server_negotiation.cc:299] Negotiation successful
0809 02:02:49.328502 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 02:02:49.329464 7093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:49.324448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:41123 (local address 127.1.192.193:58643)
0809 02:02:49.325150 (+ 702us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:49.325194 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 02:02:49.326587 (+ 1393us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:02:49.327168 (+ 581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:02:49.327174 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:02:49.327260 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:02:49.327886 (+ 626us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 02:02:49.327887 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:02:49.327929 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 02:02:49.327997 (+ 68us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:02:49.328302 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:02:49.328312 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:02:49.328317 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 02:02:49.328502 (+ 185us) client_negotiation.cc:240] Negotiation successful
0809 02:02:49.328752 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":127,"threads_started":1}
I20260809 02:02:49.330753 7091 heartbeater.cc:347] Connected to a master server at 127.1.192.254:41123
I20260809 02:02:49.330979 7091 heartbeater.cc:464] Registering TS with master...
I20260809 02:02:49.331400 7091 heartbeater.cc:511] Master 127.1.192.254:41123 requested a full tablet report, sending...
I20260809 02:02:49.331732 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 6956
I20260809 02:02:49.331851 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786240941968412-1795-0/minicluster-data/ts-0/wal/instance
I20260809 02:02:49.332141 1795 external_mini_cluster.cc:1428] Setting key 3dbbaf3ea939a92fc27810db9ac57b58
I20260809 02:02:49.333027 6889 ts_manager.cc:194] Registered new tserver with Master: f9e1088d93c24328aa5b872dd8764046 (127.1.192.193:39813)
I20260809 02:02:49.334342 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786240941968412-1795-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786240941968412-1795-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786240941968412-1795-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786240941968412-1795-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.192.194:0
--local_ip_for_outbound_sockets=127.1.192.194
--webserver_interface=127.1.192.194
--webserver_port=0
--tserver_master_addrs=127.1.192.254:41123
--builtin_ntp_servers=127.1.192.212:40641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 02:02:49.446350 7094 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:02:49.446630 7094 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:02:49.446672 7094 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:02:49.450956 7094 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:02:49.451020 7094 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:02:49.451108 7094 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.192.194
I20260809 02:02:49.455942 7094 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:40641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786240941968412-1795-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786240941968412-1795-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.192.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786240941968412-1795-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.192.194
--webserver_port=0
--tserver_master_addrs=127.1.192.254:41123
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7094
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.192.194
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786240941968412-1795-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:02:49.457099 7094 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:02:49.458566 7094 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:02:49.468423 7102 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:02:49.468423 7101 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:02:49.468466 7104 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:02:49.468904 7094 server_base.cc:1034] running on GCE node
I20260809 02:02:49.469344 7094 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:02:49.470130 7094 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:02:49.471305 7094 hybrid_clock.cc:648] HybridClock initialized: now 1786240969471275 us; error 42 us; skew 500 ppm
I20260809 02:02:49.473438 7094 webserver.cc:460] Webserver started at http://127.1.192.194:37441/ using document root <none> and password file <none>
I20260809 02:02:49.474082 7094 fs_manager.cc:359] Metadata directory not provided
I20260809 02:02:49.474148 7094 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:02:49.474376 7094 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:02:49.476609 7094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786240941968412-1795-0/minicluster-data/ts-1/data/instance:
uuid: "a66a364441e449da85f42d68336a7976"
format_stamp: "Formatted at 2026-08-09 02:02:49 on dist-test-slave-ss4n"
server_key: "0221f349b54dab49d71eb0c266cfed61"
server_key_iv: "19b634b61a9e2e050cb8ade273cfa25f"
server_key_version: "encryptionkey@0"
I20260809 02:02:49.477285 7094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786240941968412-1795-0/minicluster-data/ts-1/wal/instance:
uuid: "a66a364441e449da85f42d68336a7976"
format_stamp: "Formatted at 2026-08-09 02:02:49 on dist-test-slave-ss4n"
server_key: "0221f349b54dab49d71eb0c266cfed61"
server_key_iv: "19b634b61a9e2e050cb8ade273cfa25f"
server_key_version: "encryptionkey@0"
I20260809 02:02:49.481372 7094 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260809 02:02:49.484328 7110 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:49.485793 7094 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 02:02:49.485918 7094 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786240941968412-1795-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786240941968412-1795-0/minicluster-data/ts-1/wal
uuid: "a66a364441e449da85f42d68336a7976"
format_stamp: "Formatted at 2026-08-09 02:02:49 on dist-test-slave-ss4n"
server_key: "0221f349b54dab49d71eb0c266cfed61"
server_key_iv: "19b634b61a9e2e050cb8ade273cfa25f"
server_key_version: "encryptionkey@0"
I20260809 02:02:49.486034 7094 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786240941968412-1795-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786240941968412-1795-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786240941968412-1795-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:02:49.490661 7094 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:02:49.491340 7094 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:02:49.491530 7094 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:02:49.492100 7094 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:02:49.493014 7094 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:02:49.493068 7094 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:49.493105 7094 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:02:49.493121 7094 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:49.501472 7094 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.194:42529
I20260809 02:02:49.501561 7234 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.194:42529 every 8 connection(s)
I20260809 02:02:49.502514 7094 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786240941968412-1795-0/minicluster-data/ts-1/data/info.pb
I20260809 02:02:49.509181 6933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:49.504569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.194:52715 (local address 127.1.192.254:41123)
0809 02:02:49.504741 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:49.504745 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:49.505821 (+ 1076us) server_negotiation.cc:407] Connection header received
0809 02:02:49.507182 (+ 1361us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:49.507186 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:49.507228 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:49.507327 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:49.508672 (+ 1345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:02:49.508679 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:02:49.508685 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:02:49.508716 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:02:49.508742 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:02:49.508759 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:02:49.508844 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:49.509025 (+ 181us) server_negotiation.cc:299] Negotiation successful
0809 02:02:49.509067 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260809 02:02:49.510190 7237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:49.504890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:41123 (local address 127.1.192.194:52715)
0809 02:02:49.505670 (+ 780us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:49.505713 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 02:02:49.506905 (+ 1192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:02:49.507566 (+ 661us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:02:49.507573 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:02:49.507674 (+ 101us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:02:49.508291 (+ 617us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 02:02:49.508293 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:02:49.508336 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 02:02:49.508410 (+ 74us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:02:49.508867 (+ 457us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:02:49.508875 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:02:49.508888 (+ 13us) client_negotiation.cc:769] Sending connection context
0809 02:02:49.509048 (+ 160us) client_negotiation.cc:240] Negotiation successful
0809 02:02:49.509250 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":134,"threads_started":1}
I20260809 02:02:49.511314 7235 heartbeater.cc:347] Connected to a master server at 127.1.192.254:41123
I20260809 02:02:49.511416 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 7094
I20260809 02:02:49.511554 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786240941968412-1795-0/minicluster-data/ts-1/wal/instance
I20260809 02:02:49.511569 7235 heartbeater.cc:464] Registering TS with master...
I20260809 02:02:49.511838 1795 external_mini_cluster.cc:1428] Setting key 280bd9639f678163fd349ae84ce5c74b
I20260809 02:02:49.512024 7235 heartbeater.cc:511] Master 127.1.192.254:41123 requested a full tablet report, sending...
I20260809 02:02:49.513098 6888 ts_manager.cc:194] Registered new tserver with Master: a66a364441e449da85f42d68336a7976 (127.1.192.194:42529)
I20260809 02:02:49.514173 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786240941968412-1795-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786240941968412-1795-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786240941968412-1795-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786240941968412-1795-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.192.195:0
--local_ip_for_outbound_sockets=127.1.192.195
--webserver_interface=127.1.192.195
--webserver_port=0
--tserver_master_addrs=127.1.192.254:41123
--builtin_ntp_servers=127.1.192.212:40641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 02:02:49.626863 7238 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:02:49.627133 7238 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:02:49.627184 7238 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:02:49.631212 7238 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:02:49.631276 7238 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:02:49.631335 7238 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.192.195
I20260809 02:02:49.636269 7238 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:40641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786240941968412-1795-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786240941968412-1795-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.192.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786240941968412-1795-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.192.195
--webserver_port=0
--tserver_master_addrs=127.1.192.254:41123
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7238
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.192.195
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786240941968412-1795-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:02:49.637454 7238 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:02:49.638928 7238 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:02:49.649022 7250 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:02:49.649116 7247 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:02:49.649022 7244 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:02:49.649173 7238 server_base.cc:1034] running on GCE node
I20260809 02:02:49.649830 7238 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:02:49.650619 7238 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:02:49.651798 7238 hybrid_clock.cc:648] HybridClock initialized: now 1786240969651784 us; error 27 us; skew 500 ppm
I20260809 02:02:49.653860 7238 webserver.cc:460] Webserver started at http://127.1.192.195:42503/ using document root <none> and password file <none>
I20260809 02:02:49.654465 7238 fs_manager.cc:359] Metadata directory not provided
I20260809 02:02:49.654564 7238 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:02:49.654801 7238 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:02:49.656976 7238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786240941968412-1795-0/minicluster-data/ts-2/data/instance:
uuid: "954bbe9cf773432e8468746629c086e9"
format_stamp: "Formatted at 2026-08-09 02:02:49 on dist-test-slave-ss4n"
server_key: "c45805b66f7a74c08dd9eca590852756"
server_key_iv: "cd7ed41c190b29d750c498b6a06a4a3a"
server_key_version: "encryptionkey@0"
I20260809 02:02:49.657707 7238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786240941968412-1795-0/minicluster-data/ts-2/wal/instance:
uuid: "954bbe9cf773432e8468746629c086e9"
format_stamp: "Formatted at 2026-08-09 02:02:49 on dist-test-slave-ss4n"
server_key: "c45805b66f7a74c08dd9eca590852756"
server_key_iv: "cd7ed41c190b29d750c498b6a06a4a3a"
server_key_version: "encryptionkey@0"
I20260809 02:02:49.662235 7238 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260809 02:02:49.665300 7257 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:49.666630 7238 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260809 02:02:49.666796 7238 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786240941968412-1795-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786240941968412-1795-0/minicluster-data/ts-2/wal
uuid: "954bbe9cf773432e8468746629c086e9"
format_stamp: "Formatted at 2026-08-09 02:02:49 on dist-test-slave-ss4n"
server_key: "c45805b66f7a74c08dd9eca590852756"
server_key_iv: "cd7ed41c190b29d750c498b6a06a4a3a"
server_key_version: "encryptionkey@0"
I20260809 02:02:49.666929 7238 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786240941968412-1795-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786240941968412-1795-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786240941968412-1795-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:02:49.671727 7238 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:02:49.672341 7238 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:02:49.672525 7238 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:02:49.673089 7238 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:02:49.674103 7238 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:02:49.674176 7238 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:49.674249 7238 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:02:49.674301 7238 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:49.683223 7238 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.195:43547
I20260809 02:02:49.683245 7388 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.195:43547 every 8 connection(s)
I20260809 02:02:49.684183 7238 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786240941968412-1795-0/minicluster-data/ts-2/data/info.pb
I20260809 02:02:49.691020 6933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:49.686116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.195:46475 (local address 127.1.192.254:41123)
0809 02:02:49.686261 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:49.686265 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:49.687444 (+ 1179us) server_negotiation.cc:407] Connection header received
0809 02:02:49.688929 (+ 1485us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:49.688933 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:49.689042 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:49.689154 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:49.690482 (+ 1328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:02:49.690487 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:02:49.690490 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:02:49.690504 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:02:49.690528 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:02:49.690540 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:02:49.690653 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:49.690812 (+ 159us) server_negotiation.cc:299] Negotiation successful
0809 02:02:49.690862 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 02:02:49.691220 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 7238
I20260809 02:02:49.691342 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786240941968412-1795-0/minicluster-data/ts-2/wal/instance
I20260809 02:02:49.691609 1795 external_mini_cluster.cc:1428] Setting key ee722f9c45505eeaa7f3c68fbaaf0d7c
I20260809 02:02:49.692086 7393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:49.686531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:41123 (local address 127.1.192.195:46475)
0809 02:02:49.687263 (+ 732us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:49.687308 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 02:02:49.688688 (+ 1380us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:02:49.689332 (+ 644us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:02:49.689339 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:02:49.689431 (+ 92us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:02:49.690212 (+ 781us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 02:02:49.690214 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:02:49.690256 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 02:02:49.690324 (+ 68us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:02:49.690645 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:02:49.690654 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:02:49.690661 (+ 7us) client_negotiation.cc:769] Sending connection context
0809 02:02:49.690856 (+ 195us) client_negotiation.cc:240] Negotiation successful
0809 02:02:49.691123 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":118,"threads_started":1}
I20260809 02:02:49.693305 7389 heartbeater.cc:347] Connected to a master server at 127.1.192.254:41123
I20260809 02:02:49.693586 7389 heartbeater.cc:464] Registering TS with master...
I20260809 02:02:49.694068 7389 heartbeater.cc:511] Master 127.1.192.254:41123 requested a full tablet report, sending...
I20260809 02:02:49.695148 6888 ts_manager.cc:194] Registered new tserver with Master: 954bbe9cf773432e8468746629c086e9 (127.1.192.195:43547)
I20260809 02:02:49.706033 1795 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 02:02:49.709688 6933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:49.707980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35710 (local address 127.1.192.254:41123)
0809 02:02:49.708145 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:49.708149 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:49.708308 (+ 159us) server_negotiation.cc:407] Connection header received
0809 02:02:49.708444 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:49.708446 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:49.708481 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:49.708566 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:49.709307 (+ 741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:02:49.709310 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:02:49.709312 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:02:49.709323 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:02:49.709341 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:02:49.709348 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:02:49.709418 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:49.709541 (+ 123us) server_negotiation.cc:299] Negotiation successful
0809 02:02:49.709583 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260809 02:02:49.712980 6888 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35710:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 02:02:49.716038 6888 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:02:49.728397 7405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:49.725103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.194:42529 (local address 127.0.0.1:56276)
0809 02:02:49.725559 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:49.725574 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 02:02:49.725664 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:02:49.727020 (+ 1356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:02:49.727023 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:02:49.727039 (+ 16us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:02:49.727472 (+ 433us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 02:02:49.727474 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:02:49.727482 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 02:02:49.727502 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:02:49.728007 (+ 505us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:02:49.728018 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:02:49.728025 (+ 7us) client_negotiation.cc:769] Sending connection context
0809 02:02:49.728149 (+ 124us) client_negotiation.cc:240] Negotiation successful
0809 02:02:49.728220 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":96,"threads_started":1}
I20260809 02:02:49.728456 7404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:49.724913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49440 (local address 127.1.192.195:43547)
0809 02:02:49.725922 (+ 1009us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:49.725934 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:49.725963 (+ 29us) server_negotiation.cc:407] Connection header received
0809 02:02:49.726034 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:49.726040 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:49.726250 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:49.726405 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:49.727748 (+ 1343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:02:49.727754 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:02:49.727764 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:02:49.727776 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:02:49.727814 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:02:49.727847 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:02:49.727962 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:49.728197 (+ 235us) server_negotiation.cc:299] Negotiation successful
0809 02:02:49.728277 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":870,"thread_start_us":110,"threads_started":1}
I20260809 02:02:49.728701 7407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:49.725646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.193:39813 (local address 127.0.0.1:53660)
0809 02:02:49.726195 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:49.726211 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 02:02:49.726295 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:02:49.727518 (+ 1223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:02:49.727522 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:02:49.727536 (+ 14us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:02:49.728033 (+ 497us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 02:02:49.728035 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:02:49.728047 (+ 12us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 02:02:49.728069 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:02:49.728457 (+ 388us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:02:49.728460 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:02:49.728463 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 02:02:49.728539 (+ 76us) client_negotiation.cc:240] Negotiation successful
0809 02:02:49.728570 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":111,"threads_started":1}
I20260809 02:02:49.729044 7406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:49.725558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56276 (local address 127.1.192.194:42529)
0809 02:02:49.726573 (+ 1015us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:49.726585 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:49.726608 (+ 23us) server_negotiation.cc:407] Connection header received
0809 02:02:49.726662 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:49.726667 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:49.726865 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:49.727036 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:49.727736 (+ 700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:02:49.727742 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:02:49.727755 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:02:49.727768 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:02:49.727813 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:02:49.727845 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:02:49.727999 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:49.728787 (+ 788us) server_negotiation.cc:299] Negotiation successful
0809 02:02:49.728873 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":908,"thread_start_us":118,"threads_started":1}
I20260809 02:02:49.729077 7403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:49.724787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.195:43547 (local address 127.0.0.1:49440)
0809 02:02:49.725191 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:49.725244 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 02:02:49.725350 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:02:49.726487 (+ 1137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:02:49.726495 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:02:49.726577 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:02:49.727299 (+ 722us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 02:02:49.727302 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:02:49.727325 (+ 23us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 02:02:49.727606 (+ 281us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:02:49.728011 (+ 405us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:02:49.728023 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:02:49.728027 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 02:02:49.728171 (+ 144us) client_negotiation.cc:240] Negotiation successful
0809 02:02:49.728236 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":218,"mutex_wait_us":6,"thread_start_us":134,"threads_started":1}
I20260809 02:02:49.729218 7408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:49.726131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53660 (local address 127.1.192.193:39813)
0809 02:02:49.726569 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:49.726579 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:49.726602 (+ 23us) server_negotiation.cc:407] Connection header received
0809 02:02:49.726665 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:49.726673 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:49.727154 (+ 481us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:49.727320 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:49.728215 (+ 895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:02:49.728221 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:02:49.728231 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:02:49.728243 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:02:49.728285 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:02:49.728317 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:02:49.728411 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:49.728970 (+ 559us) server_negotiation.cc:299] Negotiation successful
0809 02:02:49.729033 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346,"thread_start_us":114,"threads_started":1}
I20260809 02:02:49.731880 7022 tablet_service.cc:1467] Processing CreateTablet for tablet 8e4133a3461e4c2b97d643b17bb32e50 (DEFAULT_TABLE table=test-table [id=0fc491cad9d740179c54b124aeb385a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:02:49.731935 7310 tablet_service.cc:1467] Processing CreateTablet for tablet 8e4133a3461e4c2b97d643b17bb32e50 (DEFAULT_TABLE table=test-table [id=0fc491cad9d740179c54b124aeb385a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:02:49.731935 7163 tablet_service.cc:1467] Processing CreateTablet for tablet 8e4133a3461e4c2b97d643b17bb32e50 (DEFAULT_TABLE table=test-table [id=0fc491cad9d740179c54b124aeb385a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:02:49.732975 7163 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e4133a3461e4c2b97d643b17bb32e50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:02:49.732975 7310 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e4133a3461e4c2b97d643b17bb32e50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:02:49.732985 7022 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e4133a3461e4c2b97d643b17bb32e50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:02:49.739573 7409 tablet_bootstrap.cc:492] T 8e4133a3461e4c2b97d643b17bb32e50 P f9e1088d93c24328aa5b872dd8764046: Bootstrap starting.
I20260809 02:02:49.740563 7411 tablet_bootstrap.cc:492] T 8e4133a3461e4c2b97d643b17bb32e50 P 954bbe9cf773432e8468746629c086e9: Bootstrap starting.
I20260809 02:02:49.740983 7410 tablet_bootstrap.cc:492] T 8e4133a3461e4c2b97d643b17bb32e50 P a66a364441e449da85f42d68336a7976: Bootstrap starting.
I20260809 02:02:49.742380 7409 tablet_bootstrap.cc:654] T 8e4133a3461e4c2b97d643b17bb32e50 P f9e1088d93c24328aa5b872dd8764046: Neither blocks nor log segments found. Creating new log.
I20260809 02:02:49.742745 7411 tablet_bootstrap.cc:654] T 8e4133a3461e4c2b97d643b17bb32e50 P 954bbe9cf773432e8468746629c086e9: Neither blocks nor log segments found. Creating new log.
I20260809 02:02:49.743172 7410 tablet_bootstrap.cc:654] T 8e4133a3461e4c2b97d643b17bb32e50 P a66a364441e449da85f42d68336a7976: Neither blocks nor log segments found. Creating new log.
I20260809 02:02:49.743381 7409 log.cc:824] T 8e4133a3461e4c2b97d643b17bb32e50 P f9e1088d93c24328aa5b872dd8764046: Log is configured to *not* fsync() on all Append() calls
I20260809 02:02:49.743567 7411 log.cc:824] T 8e4133a3461e4c2b97d643b17bb32e50 P 954bbe9cf773432e8468746629c086e9: Log is configured to *not* fsync() on all Append() calls
I20260809 02:02:49.743933 7410 log.cc:824] T 8e4133a3461e4c2b97d643b17bb32e50 P a66a364441e449da85f42d68336a7976: Log is configured to *not* fsync() on all Append() calls
I20260809 02:02:49.745468 7409 tablet_bootstrap.cc:492] T 8e4133a3461e4c2b97d643b17bb32e50 P f9e1088d93c24328aa5b872dd8764046: No bootstrap required, opened a new log
I20260809 02:02:49.745690 7410 tablet_bootstrap.cc:492] T 8e4133a3461e4c2b97d643b17bb32e50 P a66a364441e449da85f42d68336a7976: No bootstrap required, opened a new log
I20260809 02:02:49.745738 7409 ts_tablet_manager.cc:1397] T 8e4133a3461e4c2b97d643b17bb32e50 P f9e1088d93c24328aa5b872dd8764046: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.001s
I20260809 02:02:49.745883 7410 ts_tablet_manager.cc:1397] T 8e4133a3461e4c2b97d643b17bb32e50 P a66a364441e449da85f42d68336a7976: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260809 02:02:49.749666 7410 raft_consensus.cc:348] T 8e4133a3461e4c2b97d643b17bb32e50 P a66a364441e449da85f42d68336a7976 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "954bbe9cf773432e8468746629c086e9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 43547 } } peers { permanent_uuid: "f9e1088d93c24328aa5b872dd8764046" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 39813 } } peers { permanent_uuid: "a66a364441e449da85f42d68336a7976" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 42529 } }
I20260809 02:02:49.749989 7410 raft_consensus.cc:374] T 8e4133a3461e4c2b97d643b17bb32e50 P a66a364441e449da85f42d68336a7976 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:02:49.750051 7410 raft_consensus.cc:729] T 8e4133a3461e4c2b97d643b17bb32e50 P a66a364441e449da85f42d68336a7976 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a66a364441e449da85f42d68336a7976, State: Initialized, Role: FOLLOWER
I20260809 02:02:49.749922 7409 raft_consensus.cc:348] T 8e4133a3461e4c2b97d643b17bb32e50 P f9e1088d93c24328aa5b872dd8764046 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "954bbe9cf773432e8468746629c086e9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 43547 } } peers { permanent_uuid: "f9e1088d93c24328aa5b872dd8764046" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 39813 } } peers { permanent_uuid: "a66a364441e449da85f42d68336a7976" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 42529 } }
I20260809 02:02:49.750206 7409 raft_consensus.cc:374] T 8e4133a3461e4c2b97d643b17bb32e50 P f9e1088d93c24328aa5b872dd8764046 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:02:49.750303 7409 raft_consensus.cc:729] T 8e4133a3461e4c2b97d643b17bb32e50 P f9e1088d93c24328aa5b872dd8764046 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9e1088d93c24328aa5b872dd8764046, State: Initialized, Role: FOLLOWER
I20260809 02:02:49.750550 7410 consensus_queue.cc:260] T 8e4133a3461e4c2b97d643b17bb32e50 P a66a364441e449da85f42d68336a7976 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "954bbe9cf773432e8468746629c086e9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 43547 } } peers { permanent_uuid: "f9e1088d93c24328aa5b872dd8764046" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 39813 } } peers { permanent_uuid: "a66a364441e449da85f42d68336a7976" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 42529 } }
I20260809 02:02:49.750773 7409 consensus_queue.cc:260] T 8e4133a3461e4c2b97d643b17bb32e50 P f9e1088d93c24328aa5b872dd8764046 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "954bbe9cf773432e8468746629c086e9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 43547 } } peers { permanent_uuid: "f9e1088d93c24328aa5b872dd8764046" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 39813 } } peers { permanent_uuid: "a66a364441e449da85f42d68336a7976" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 42529 } }
I20260809 02:02:49.751355 7235 heartbeater.cc:503] Master 127.1.192.254:41123 was elected leader, sending a full tablet report...
I20260809 02:02:49.751488 7091 heartbeater.cc:503] Master 127.1.192.254:41123 was elected leader, sending a full tablet report...
I20260809 02:02:49.751703 7410 ts_tablet_manager.cc:1428] T 8e4133a3461e4c2b97d643b17bb32e50 P a66a364441e449da85f42d68336a7976: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260809 02:02:49.752301 7409 ts_tablet_manager.cc:1428] T 8e4133a3461e4c2b97d643b17bb32e50 P f9e1088d93c24328aa5b872dd8764046: Time spent starting tablet: real 0.006s user 0.003s sys 0.003s
W20260809 02:02:49.753530 7236 tablet.cc:2369] T 8e4133a3461e4c2b97d643b17bb32e50 P a66a364441e449da85f42d68336a7976: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 02:02:49.754159 7411 tablet_bootstrap.cc:492] T 8e4133a3461e4c2b97d643b17bb32e50 P 954bbe9cf773432e8468746629c086e9: No bootstrap required, opened a new log
I20260809 02:02:49.754408 7411 ts_tablet_manager.cc:1397] T 8e4133a3461e4c2b97d643b17bb32e50 P 954bbe9cf773432e8468746629c086e9: Time spent bootstrapping tablet: real 0.014s user 0.005s sys 0.000s
I20260809 02:02:49.758610 7411 raft_consensus.cc:348] T 8e4133a3461e4c2b97d643b17bb32e50 P 954bbe9cf773432e8468746629c086e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "954bbe9cf773432e8468746629c086e9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 43547 } } peers { permanent_uuid: "f9e1088d93c24328aa5b872dd8764046" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 39813 } } peers { permanent_uuid: "a66a364441e449da85f42d68336a7976" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 42529 } }
I20260809 02:02:49.758848 7411 raft_consensus.cc:374] T 8e4133a3461e4c2b97d643b17bb32e50 P 954bbe9cf773432e8468746629c086e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:02:49.758881 7411 raft_consensus.cc:729] T 8e4133a3461e4c2b97d643b17bb32e50 P 954bbe9cf773432e8468746629c086e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 954bbe9cf773432e8468746629c086e9, State: Initialized, Role: FOLLOWER
I20260809 02:02:49.759235 7411 consensus_queue.cc:260] T 8e4133a3461e4c2b97d643b17bb32e50 P 954bbe9cf773432e8468746629c086e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "954bbe9cf773432e8468746629c086e9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 43547 } } peers { permanent_uuid: "f9e1088d93c24328aa5b872dd8764046" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 39813 } } peers { permanent_uuid: "a66a364441e449da85f42d68336a7976" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 42529 } }
I20260809 02:02:49.759894 7389 heartbeater.cc:503] Master 127.1.192.254:41123 was elected leader, sending a full tablet report...
I20260809 02:02:49.760157 7411 ts_tablet_manager.cc:1428] T 8e4133a3461e4c2b97d643b17bb32e50 P 954bbe9cf773432e8468746629c086e9: Time spent starting tablet: real 0.006s user 0.002s sys 0.004s
W20260809 02:02:49.823421 7092 tablet.cc:2369] T 8e4133a3461e4c2b97d643b17bb32e50 P f9e1088d93c24328aa5b872dd8764046: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 02:02:49.935215 7391 tablet.cc:2369] T 8e4133a3461e4c2b97d643b17bb32e50 P 954bbe9cf773432e8468746629c086e9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 02:02:50.108866 7417 raft_consensus.cc:482] T 8e4133a3461e4c2b97d643b17bb32e50 P f9e1088d93c24328aa5b872dd8764046 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:02:50.109086 7417 raft_consensus.cc:504] T 8e4133a3461e4c2b97d643b17bb32e50 P f9e1088d93c24328aa5b872dd8764046 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "954bbe9cf773432e8468746629c086e9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 43547 } } peers { permanent_uuid: "f9e1088d93c24328aa5b872dd8764046" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 39813 } } peers { permanent_uuid: "a66a364441e449da85f42d68336a7976" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 42529 } }
I20260809 02:02:50.110366 7417 leader_election.cc:288] T 8e4133a3461e4c2b97d643b17bb32e50 P f9e1088d93c24328aa5b872dd8764046 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 954bbe9cf773432e8468746629c086e9 (127.1.192.195:43547), a66a364441e449da85f42d68336a7976 (127.1.192.194:42529)
I20260809 02:02:50.112588 7404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:50.110723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.193:53623 (local address 127.1.192.195:43547)
0809 02:02:50.110999 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:50.111004 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:50.111023 (+ 19us) server_negotiation.cc:407] Connection header received
0809 02:02:50.111155 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:50.111158 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:50.111199 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:50.111285 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:50.112101 (+ 816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:02:50.112107 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:02:50.112113 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:02:50.112126 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:02:50.112147 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:02:50.112158 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:02:50.112262 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:50.112378 (+ 116us) server_negotiation.cc:299] Negotiation successful
0809 02:02:50.112414 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176}
I20260809 02:02:50.112658 7422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:50.110611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.195:43547 (local address 127.1.192.193:53623)
0809 02:02:50.110950 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:50.110962 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:02:50.111046 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:02:50.111335 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:02:50.111338 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:02:50.111353 (+ 15us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:02:50.111913 (+ 560us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 02:02:50.111916 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:02:50.111929 (+ 13us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 02:02:50.111965 (+ 36us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:02:50.112272 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:02:50.112278 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:02:50.112282 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 02:02:50.112374 (+ 92us) client_negotiation.cc:240] Negotiation successful
0809 02:02:50.112415 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":95,"threads_started":1}
I20260809 02:02:50.112726 7406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:50.110751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.193:56127 (local address 127.1.192.194:42529)
0809 02:02:50.110876 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:50.110881 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:50.111170 (+ 289us) server_negotiation.cc:407] Connection header received
0809 02:02:50.111323 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:50.111326 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:50.111366 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:50.111454 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:50.112326 (+ 872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:02:50.112330 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:02:50.112337 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:02:50.112352 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:02:50.112374 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:02:50.112384 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:02:50.112467 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:50.112559 (+ 92us) server_negotiation.cc:299] Negotiation successful
0809 02:02:50.112598 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 02:02:50.113231 7421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:50.110614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.194:42529 (local address 127.1.192.193:56127)
0809 02:02:50.111028 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:50.111043 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 02:02:50.111190 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:02:50.111479 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:02:50.111482 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:02:50.111493 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:02:50.112134 (+ 641us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 02:02:50.112137 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:02:50.112168 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 02:02:50.112197 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:02:50.112471 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:02:50.112474 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:02:50.112477 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 02:02:50.112560 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 02:02:50.112592 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":351,"mutex_wait_us":80,"thread_start_us":68,"threads_started":1}
I20260809 02:02:50.113451 7339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e4133a3461e4c2b97d643b17bb32e50" candidate_uuid: "f9e1088d93c24328aa5b872dd8764046" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "954bbe9cf773432e8468746629c086e9" is_pre_election: true
I20260809 02:02:50.113837 7339 raft_consensus.cc:2454] T 8e4133a3461e4c2b97d643b17bb32e50 P 954bbe9cf773432e8468746629c086e9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f9e1088d93c24328aa5b872dd8764046 in term 0.
I20260809 02:02:50.113897 7185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e4133a3461e4c2b97d643b17bb32e50" candidate_uuid: "f9e1088d93c24328aa5b872dd8764046" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a66a364441e449da85f42d68336a7976" is_pre_election: true
I20260809 02:02:50.114199 7185 raft_consensus.cc:2454] T 8e4133a3461e4c2b97d643b17bb32e50 P a66a364441e449da85f42d68336a7976 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f9e1088d93c24328aa5b872dd8764046 in term 0.
I20260809 02:02:50.114373 6976 leader_election.cc:302] T 8e4133a3461e4c2b97d643b17bb32e50 P f9e1088d93c24328aa5b872dd8764046 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 954bbe9cf773432e8468746629c086e9, f9e1088d93c24328aa5b872dd8764046; no voters:
I20260809 02:02:50.114650 7417 raft_consensus.cc:2781] T 8e4133a3461e4c2b97d643b17bb32e50 P f9e1088d93c24328aa5b872dd8764046 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:02:50.114737 7417 raft_consensus.cc:482] T 8e4133a3461e4c2b97d643b17bb32e50 P f9e1088d93c24328aa5b872dd8764046 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:02:50.114768 7417 raft_consensus.cc:3037] T 8e4133a3461e4c2b97d643b17bb32e50 P f9e1088d93c24328aa5b872dd8764046 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:02:50.115863 7417 raft_consensus.cc:504] T 8e4133a3461e4c2b97d643b17bb32e50 P f9e1088d93c24328aa5b872dd8764046 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "954bbe9cf773432e8468746629c086e9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 43547 } } peers { permanent_uuid: "f9e1088d93c24328aa5b872dd8764046" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 39813 } } peers { permanent_uuid: "a66a364441e449da85f42d68336a7976" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 42529 } }
I20260809 02:02:50.116237 7417 leader_election.cc:288] T 8e4133a3461e4c2b97d643b17bb32e50 P f9e1088d93c24328aa5b872dd8764046 [CANDIDATE]: Term 1 election: Requested vote from peers 954bbe9cf773432e8468746629c086e9 (127.1.192.195:43547), a66a364441e449da85f42d68336a7976 (127.1.192.194:42529)
I20260809 02:02:50.116561 7339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e4133a3461e4c2b97d643b17bb32e50" candidate_uuid: "f9e1088d93c24328aa5b872dd8764046" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "954bbe9cf773432e8468746629c086e9"
I20260809 02:02:50.116588 7185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e4133a3461e4c2b97d643b17bb32e50" candidate_uuid: "f9e1088d93c24328aa5b872dd8764046" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a66a364441e449da85f42d68336a7976"
I20260809 02:02:50.116725 7339 raft_consensus.cc:3037] T 8e4133a3461e4c2b97d643b17bb32e50 P 954bbe9cf773432e8468746629c086e9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:02:50.116735 7185 raft_consensus.cc:3037] T 8e4133a3461e4c2b97d643b17bb32e50 P a66a364441e449da85f42d68336a7976 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:02:50.117848 7185 raft_consensus.cc:2454] T 8e4133a3461e4c2b97d643b17bb32e50 P a66a364441e449da85f42d68336a7976 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9e1088d93c24328aa5b872dd8764046 in term 1.
I20260809 02:02:50.117867 7339 raft_consensus.cc:2454] T 8e4133a3461e4c2b97d643b17bb32e50 P 954bbe9cf773432e8468746629c086e9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9e1088d93c24328aa5b872dd8764046 in term 1.
I20260809 02:02:50.118184 6976 leader_election.cc:302] T 8e4133a3461e4c2b97d643b17bb32e50 P f9e1088d93c24328aa5b872dd8764046 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 954bbe9cf773432e8468746629c086e9, f9e1088d93c24328aa5b872dd8764046; no voters:
I20260809 02:02:50.118425 7417 raft_consensus.cc:2781] T 8e4133a3461e4c2b97d643b17bb32e50 P f9e1088d93c24328aa5b872dd8764046 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:02:50.118649 7417 raft_consensus.cc:686] T 8e4133a3461e4c2b97d643b17bb32e50 P f9e1088d93c24328aa5b872dd8764046 [term 1 LEADER]: Becoming Leader. State: Replica: f9e1088d93c24328aa5b872dd8764046, State: Running, Role: LEADER
I20260809 02:02:50.118965 7417 consensus_queue.cc:237] T 8e4133a3461e4c2b97d643b17bb32e50 P f9e1088d93c24328aa5b872dd8764046 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "954bbe9cf773432e8468746629c086e9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 43547 } } peers { permanent_uuid: "f9e1088d93c24328aa5b872dd8764046" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 39813 } } peers { permanent_uuid: "a66a364441e449da85f42d68336a7976" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 42529 } }
I20260809 02:02:50.122514 6889 catalog_manager.cc:5495] T 8e4133a3461e4c2b97d643b17bb32e50 P f9e1088d93c24328aa5b872dd8764046 reported cstate change: term changed from 0 to 1, leader changed from <none> to f9e1088d93c24328aa5b872dd8764046 (127.1.192.193). New cstate: current_term: 1 leader_uuid: "f9e1088d93c24328aa5b872dd8764046" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "954bbe9cf773432e8468746629c086e9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 43547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f9e1088d93c24328aa5b872dd8764046" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 39813 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a66a364441e449da85f42d68336a7976" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 42529 } health_report { overall_health: UNKNOWN } } }
I20260809 02:02:50.183521 7408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:50.181745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53666 (local address 127.1.192.193:39813)
0809 02:02:50.181943 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:50.181950 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:50.182032 (+ 82us) server_negotiation.cc:407] Connection header received
0809 02:02:50.182222 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:50.182225 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:50.182288 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:50.182391 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:50.183126 (+ 735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:02:50.183129 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:02:50.183131 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:02:50.183143 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:02:50.183162 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:02:50.183170 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:02:50.183246 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:50.183315 (+ 69us) server_negotiation.cc:299] Negotiation successful
0809 02:02:50.183347 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260809 02:02:50.189941 7185 raft_consensus.cc:1261] T 8e4133a3461e4c2b97d643b17bb32e50 P a66a364441e449da85f42d68336a7976 [term 1 FOLLOWER]: Refusing update from remote peer f9e1088d93c24328aa5b872dd8764046: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:02:50.189941 7339 raft_consensus.cc:1261] T 8e4133a3461e4c2b97d643b17bb32e50 P 954bbe9cf773432e8468746629c086e9 [term 1 FOLLOWER]: Refusing update from remote peer f9e1088d93c24328aa5b872dd8764046: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:02:50.190773 7423 consensus_queue.cc:1035] T 8e4133a3461e4c2b97d643b17bb32e50 P f9e1088d93c24328aa5b872dd8764046 [LEADER]: Connected to new peer: Peer: permanent_uuid: "954bbe9cf773432e8468746629c086e9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 43547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:02:50.190971 7417 consensus_queue.cc:1035] T 8e4133a3461e4c2b97d643b17bb32e50 P f9e1088d93c24328aa5b872dd8764046 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a66a364441e449da85f42d68336a7976" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 42529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:02:50.200855 7429 mvcc.cc:205] Tried to move back new op lower bound from 7316443013889458176 to 7316443013609717760. Current Snapshot: MvccSnapshot[applied={T|T < 7316443013889458176}]
I20260809 02:02:50.201067 7430 mvcc.cc:205] Tried to move back new op lower bound from 7316443013889458176 to 7316443013609717760. Current Snapshot: MvccSnapshot[applied={T|T < 7316443013889458176 or (T in {7316443013889458176})}]
I20260809 02:02:50.206549 6889 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35710:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:02:50.206761 6889 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35710:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:02:50.209352 6889 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 169f595f250a404883a589b70d084275: Sending DeleteTablet for 3 replicas of tablet 8e4133a3461e4c2b97d643b17bb32e50
I20260809 02:02:50.210296 7163 tablet_service.cc:1514] Processing DeleteTablet for tablet 8e4133a3461e4c2b97d643b17bb32e50 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:02:50 UTC) from {username='slave'} at 127.0.0.1:56276
I20260809 02:02:50.210438 7310 tablet_service.cc:1514] Processing DeleteTablet for tablet 8e4133a3461e4c2b97d643b17bb32e50 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:02:50 UTC) from {username='slave'} at 127.0.0.1:49440
I20260809 02:02:50.210551 7022 tablet_service.cc:1514] Processing DeleteTablet for tablet 8e4133a3461e4c2b97d643b17bb32e50 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:02:50 UTC) from {username='slave'} at 127.0.0.1:53660
I20260809 02:02:50.211030 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 6956
I20260809 02:02:50.211687 7437 tablet_replica.cc:330] T 8e4133a3461e4c2b97d643b17bb32e50 P a66a364441e449da85f42d68336a7976: stopping tablet replica
I20260809 02:02:50.211992 7437 raft_consensus.cc:2229] T 8e4133a3461e4c2b97d643b17bb32e50 P a66a364441e449da85f42d68336a7976 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:02:50.212133 7438 tablet_replica.cc:330] T 8e4133a3461e4c2b97d643b17bb32e50 P 954bbe9cf773432e8468746629c086e9: stopping tablet replica
I20260809 02:02:50.212347 7437 raft_consensus.cc:2258] T 8e4133a3461e4c2b97d643b17bb32e50 P a66a364441e449da85f42d68336a7976 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:02:50.212407 7438 raft_consensus.cc:2229] T 8e4133a3461e4c2b97d643b17bb32e50 P 954bbe9cf773432e8468746629c086e9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:02:50.212682 7438 raft_consensus.cc:2258] T 8e4133a3461e4c2b97d643b17bb32e50 P 954bbe9cf773432e8468746629c086e9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:02:50.214272 7437 ts_tablet_manager.cc:1905] T 8e4133a3461e4c2b97d643b17bb32e50 P a66a364441e449da85f42d68336a7976: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:02:50.214376 7438 ts_tablet_manager.cc:1905] T 8e4133a3461e4c2b97d643b17bb32e50 P 954bbe9cf773432e8468746629c086e9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:02:50.216840 7437 ts_tablet_manager.cc:1918] T 8e4133a3461e4c2b97d643b17bb32e50 P a66a364441e449da85f42d68336a7976: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:02:50.216966 7437 log.cc:1196] T 8e4133a3461e4c2b97d643b17bb32e50 P a66a364441e449da85f42d68336a7976: Deleting WAL directory at /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786240941968412-1795-0/minicluster-data/ts-1/wal/wals/8e4133a3461e4c2b97d643b17bb32e50
I20260809 02:02:50.217262 7437 ts_tablet_manager.cc:1939] T 8e4133a3461e4c2b97d643b17bb32e50 P a66a364441e449da85f42d68336a7976: Deleting consensus metadata
I20260809 02:02:50.217815 7438 ts_tablet_manager.cc:1918] T 8e4133a3461e4c2b97d643b17bb32e50 P 954bbe9cf773432e8468746629c086e9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:02:50.217924 7438 log.cc:1196] T 8e4133a3461e4c2b97d643b17bb32e50 P 954bbe9cf773432e8468746629c086e9: Deleting WAL directory at /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786240941968412-1795-0/minicluster-data/ts-2/wal/wals/8e4133a3461e4c2b97d643b17bb32e50
I20260809 02:02:50.218289 7438 ts_tablet_manager.cc:1939] T 8e4133a3461e4c2b97d643b17bb32e50 P 954bbe9cf773432e8468746629c086e9: Deleting consensus metadata
I20260809 02:02:50.218555 6873 catalog_manager.cc:4841] TS a66a364441e449da85f42d68336a7976 (127.1.192.194:42529): tablet 8e4133a3461e4c2b97d643b17bb32e50 (table test-table [id=0fc491cad9d740179c54b124aeb385a6]) successfully deleted
I20260809 02:02:50.219494 6875 catalog_manager.cc:4841] TS 954bbe9cf773432e8468746629c086e9 (127.1.192.195:43547): tablet 8e4133a3461e4c2b97d643b17bb32e50 (table test-table [id=0fc491cad9d740179c54b124aeb385a6]) successfully deleted
W20260809 02:02:50.221812 6875 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.192.193:39813 (error 108)
I20260809 02:02:50.222026 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 7094
I20260809 02:02:50.223294 7403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:50.222750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.193:39813 (local address 127.0.0.1:53668)
0809 02:02:50.223093 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:50.223188 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.193:39813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":236}
W20260809 02:02:50.223984 6875 catalog_manager.cc:4584] TS f9e1088d93c24328aa5b872dd8764046 (127.1.192.193:39813): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8e4133a3461e4c2b97d643b17bb32e50: Network error: Client connection negotiation failed: client connection to 127.1.192.193:39813: connect: Connection refused (error 111)
I20260809 02:02:50.229121 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 7238
I20260809 02:02:50.239046 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 6856
2026-08-09T02:02:50Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (1389 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260809 02:02:50.262799 1795 mini_oidc.cc:152] Starting JWKS server
I20260809 02:02:50.264526 1795 webserver.cc:460] Webserver started at https://127.0.0.1:41163/ using document root <none> and password file <none>
I20260809 02:02:50.264709 1795 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 02:02:50.265004 1795 webserver.cc:460] Webserver started at http://127.0.0.1:39239/ using document root <none> and password file <none>
2026-08-09T02:02:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:02:50Z Disabled control of system clock
I20260809 02:02:50.275831 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786240941968412-1795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786240941968412-1795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786240941968412-1795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.192.254:39747
--webserver_interface=127.1.192.254
--webserver_port=0
--builtin_ntp_servers=127.1.192.212:36323
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.192.254:39747
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41163/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786240941968412-1795-0/testchainca.cert with env {}
W20260809 02:02:50.392095 7446 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:02:50.392447 7446 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:02:50.392544 7446 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:02:50.396701 7446 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:02:50.396814 7446 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:02:50.396847 7446 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:02:50.396884 7446 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:02:50.396920 7446 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 02:02:50.396952 7446 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 02:02:50.396982 7446 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:41163/jwks.json
I20260809 02:02:50.402566 7446 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:36323
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786240941968412-1795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786240941968412-1795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.192.254:39747
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.192.254:39747
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41163/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.192.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786240941968412-1795-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7446
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786240941968412-1795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:02:50.403716 7446 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:02:50.404778 7446 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:02:50.413470 7452 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:02:50.413499 7451 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:02:50.413692 7446 server_base.cc:1034] running on GCE node
W20260809 02:02:50.413524 7454 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:02:50.414369 7446 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:02:50.415520 7446 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:02:50.416754 7446 hybrid_clock.cc:648] HybridClock initialized: now 1786240970416746 us; error 33 us; skew 500 ppm
I20260809 02:02:50.418958 7446 webserver.cc:460] Webserver started at http://127.1.192.254:38177/ using document root <none> and password file <none>
I20260809 02:02:50.419607 7446 fs_manager.cc:359] Metadata directory not provided
I20260809 02:02:50.419730 7446 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:02:50.419971 7446 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:02:50.422454 7446 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786240941968412-1795-0/minicluster-data/master-0/data/instance:
uuid: "e7d6d2fcfe254d10b20be9dfba6b71f6"
format_stamp: "Formatted at 2026-08-09 02:02:50 on dist-test-slave-ss4n"
server_key: "3fd72ffa2628279177a66a8e0b203ebe"
server_key_iv: "6090932b6a03e18c0d7b275a3a6286f0"
server_key_version: "encryptionkey@0"
I20260809 02:02:50.423169 7446 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786240941968412-1795-0/minicluster-data/master-0/wal/instance:
uuid: "e7d6d2fcfe254d10b20be9dfba6b71f6"
format_stamp: "Formatted at 2026-08-09 02:02:50 on dist-test-slave-ss4n"
server_key: "3fd72ffa2628279177a66a8e0b203ebe"
server_key_iv: "6090932b6a03e18c0d7b275a3a6286f0"
server_key_version: "encryptionkey@0"
I20260809 02:02:50.427438 7446 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260809 02:02:50.430241 7460 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:50.431613 7446 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 02:02:50.431773 7446 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786240941968412-1795-0/minicluster-data/master-0/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786240941968412-1795-0/minicluster-data/master-0/wal
uuid: "e7d6d2fcfe254d10b20be9dfba6b71f6"
format_stamp: "Formatted at 2026-08-09 02:02:50 on dist-test-slave-ss4n"
server_key: "3fd72ffa2628279177a66a8e0b203ebe"
server_key_iv: "6090932b6a03e18c0d7b275a3a6286f0"
server_key_version: "encryptionkey@0"
I20260809 02:02:50.431913 7446 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786240941968412-1795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786240941968412-1795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786240941968412-1795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:02:50.450673 7446 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:02:50.451481 7446 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:02:50.451822 7446 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:02:50.459494 7446 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.254:39747
I20260809 02:02:50.459590 7518 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.254:39747 every 8 connection(s)
I20260809 02:02:50.460506 7446 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
I20260809 02:02:50.463033 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 7446
I20260809 02:02:50.463168 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786240941968412-1795-0/minicluster-data/master-0/wal/instance
I20260809 02:02:50.463476 1795 external_mini_cluster.cc:1428] Setting key 15fd05d00c020dbb5d8c40a4210a1494
I20260809 02:02:50.464395 7519 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:02:50.470386 7519 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e7d6d2fcfe254d10b20be9dfba6b71f6: Bootstrap starting.
I20260809 02:02:50.472815 7522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:50.464869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45604 (local address 127.1.192.254:39747)
0809 02:02:50.465644 (+ 775us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:50.465662 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:50.465700 (+ 38us) server_negotiation.cc:407] Connection header received
0809 02:02:50.466380 (+ 680us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:50.466397 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:50.466788 (+ 391us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:50.467116 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:50.467531 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:50.468472 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:50.469385 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:50.469923 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:50.470102 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:50.471052 (+ 950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:02:50.471057 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:02:50.471095 (+ 38us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:02:50.471489 (+ 394us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:02:50.471546 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:02:50.471566 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:02:50.471691 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:50.471852 (+ 161us) server_negotiation.cc:299] Negotiation successful
0809 02:02:50.472123 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":290,"thread_start_us":132,"threads_started":1}
I20260809 02:02:50.472880 7519 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e7d6d2fcfe254d10b20be9dfba6b71f6: Neither blocks nor log segments found. Creating new log.
I20260809 02:02:50.473836 7519 log.cc:824] T 00000000000000000000000000000000 P e7d6d2fcfe254d10b20be9dfba6b71f6: Log is configured to *not* fsync() on all Append() calls
I20260809 02:02:50.476101 1795 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 02:02:50.476203 7519 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e7d6d2fcfe254d10b20be9dfba6b71f6: No bootstrap required, opened a new log
I20260809 02:02:50.480196 7519 raft_consensus.cc:348] T 00000000000000000000000000000000 P e7d6d2fcfe254d10b20be9dfba6b71f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7d6d2fcfe254d10b20be9dfba6b71f6" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 39747 } }
I20260809 02:02:50.480435 7519 raft_consensus.cc:374] T 00000000000000000000000000000000 P e7d6d2fcfe254d10b20be9dfba6b71f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:02:50.480505 7519 raft_consensus.cc:729] T 00000000000000000000000000000000 P e7d6d2fcfe254d10b20be9dfba6b71f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7d6d2fcfe254d10b20be9dfba6b71f6, State: Initialized, Role: FOLLOWER
I20260809 02:02:50.480964 7519 consensus_queue.cc:260] T 00000000000000000000000000000000 P e7d6d2fcfe254d10b20be9dfba6b71f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7d6d2fcfe254d10b20be9dfba6b71f6" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 39747 } }
I20260809 02:02:50.481086 7519 raft_consensus.cc:388] T 00000000000000000000000000000000 P e7d6d2fcfe254d10b20be9dfba6b71f6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:02:50.481161 7519 raft_consensus.cc:482] T 00000000000000000000000000000000 P e7d6d2fcfe254d10b20be9dfba6b71f6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:02:50.481287 7519 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e7d6d2fcfe254d10b20be9dfba6b71f6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:02:50.482512 7519 raft_consensus.cc:504] T 00000000000000000000000000000000 P e7d6d2fcfe254d10b20be9dfba6b71f6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7d6d2fcfe254d10b20be9dfba6b71f6" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 39747 } }
I20260809 02:02:50.482874 7519 leader_election.cc:302] T 00000000000000000000000000000000 P e7d6d2fcfe254d10b20be9dfba6b71f6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e7d6d2fcfe254d10b20be9dfba6b71f6; no voters:
I20260809 02:02:50.483217 7519 leader_election.cc:288] T 00000000000000000000000000000000 P e7d6d2fcfe254d10b20be9dfba6b71f6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:02:50.483376 7524 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e7d6d2fcfe254d10b20be9dfba6b71f6 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:02:50.483700 7524 raft_consensus.cc:686] T 00000000000000000000000000000000 P e7d6d2fcfe254d10b20be9dfba6b71f6 [term 1 LEADER]: Becoming Leader. State: Replica: e7d6d2fcfe254d10b20be9dfba6b71f6, State: Running, Role: LEADER
I20260809 02:02:50.484014 7524 consensus_queue.cc:237] T 00000000000000000000000000000000 P e7d6d2fcfe254d10b20be9dfba6b71f6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7d6d2fcfe254d10b20be9dfba6b71f6" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 39747 } }
I20260809 02:02:50.484529 7519 sys_catalog.cc:564] T 00000000000000000000000000000000 P e7d6d2fcfe254d10b20be9dfba6b71f6 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:02:50.485695 7526 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7d6d2fcfe254d10b20be9dfba6b71f6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e7d6d2fcfe254d10b20be9dfba6b71f6. Latest consensus state: current_term: 1 leader_uuid: "e7d6d2fcfe254d10b20be9dfba6b71f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7d6d2fcfe254d10b20be9dfba6b71f6" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 39747 } } }
I20260809 02:02:50.485831 7526 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7d6d2fcfe254d10b20be9dfba6b71f6 [sys.catalog]: This master's current role is: LEADER
I20260809 02:02:50.486099 7525 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7d6d2fcfe254d10b20be9dfba6b71f6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e7d6d2fcfe254d10b20be9dfba6b71f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7d6d2fcfe254d10b20be9dfba6b71f6" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 39747 } } }
I20260809 02:02:50.486192 7534 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:02:50.486210 7525 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7d6d2fcfe254d10b20be9dfba6b71f6 [sys.catalog]: This master's current role is: LEADER
I20260809 02:02:50.489635 7534 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:02:50.495527 7534 catalog_manager.cc:1347] Generated new cluster ID: 5fef94562c7d4704b5ac802488b26c68
I20260809 02:02:50.495601 7534 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:02:50.515730 7534 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:02:50.517071 7534 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:02:50.526369 7534 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e7d6d2fcfe254d10b20be9dfba6b71f6: Generated new TSK 0
I20260809 02:02:50.527197 7534 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260809 02:02:50.558918 7552 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260809 02:02:50.559123 7552 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260809 02:02:50.559794 7522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:50.541562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45614 (local address 127.1.192.254:39747)
0809 02:02:50.541706 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:50.541711 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:50.541890 (+ 179us) server_negotiation.cc:407] Connection header received
0809 02:02:50.542088 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:50.542091 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:50.542162 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:50.542287 (+ 125us) server_negotiation.cc:226] Negotiated authn=JWT
0809 02:02:50.542543 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:50.543371 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:50.544445 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:50.544783 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:50.544887 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:50.545251 (+ 364us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 02:02:50.559339 (+ 14088us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41163/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0809 02:02:50.559639 (+ 300us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:45614: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41163/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":52}
W20260809 02:02:50.559880 7522 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:45614: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41163/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260809 02:02:50.561105 7551 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:02:50.541432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:39747 (local address 127.0.0.1:45614)
0809 02:02:50.541807 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:50.541820 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:02:50.541929 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:02:50.542316 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:02:50.542319 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:02:50.542326 (+ 7us) client_negotiation.cc:189] Negotiated authn=JWT
0809 02:02:50.542431 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:50.542436 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:50.543601 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:50.543607 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:50.544321 (+ 714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:50.544330 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:50.544904 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:50.544906 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:50.544977 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:50.545045 (+ 68us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 02:02:50.559591 (+ 14546us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41163/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0809 02:02:50.559806 (+ 215us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.1.192.254:39747: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41163/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":143,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.1.192.254:39747: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41163/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260809 02:02:50.562062 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 7446
2026-08-09T02:02:50Z chronyd exiting
I20260809 02:02:50.601096 1795 test_util.cc:175] -----------------------------------------------
I20260809 02:02:50.601233 1795 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786240941968412-1795-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (340 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260809 02:02:50.602523 1795 mini_oidc.cc:152] Starting JWKS server
I20260809 02:02:50.603406 1795 webserver.cc:460] Webserver started at https://127.0.0.1:38815/ using document root <none> and password file <none>
I20260809 02:02:50.603538 1795 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 02:02:50.603814 1795 webserver.cc:460] Webserver started at http://127.0.0.1:34461/ using document root <none> and password file <none>
2026-08-09T02:02:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:02:50Z Disabled control of system clock
I20260809 02:02:50.614073 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786240941968412-1795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786240941968412-1795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786240941968412-1795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.192.254:35003
--webserver_interface=127.1.192.254
--webserver_port=0
--builtin_ntp_servers=127.1.192.212:43043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.192.254:35003
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38815/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786240941968412-1795-0/testchainca.cert with env {}
W20260809 02:02:50.729808 7559 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:02:50.730109 7559 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:02:50.730154 7559 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:02:50.734306 7559 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:02:50.734380 7559 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:02:50.734395 7559 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:02:50.734413 7559 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:02:50.734429 7559 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 02:02:50.734445 7559 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 02:02:50.734459 7559 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:38815/jwks.json
I20260809 02:02:50.740000 7559 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:43043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786240941968412-1795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786240941968412-1795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.192.254:35003
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.192.254:35003
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38815/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.192.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786240941968412-1795-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7559
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786240941968412-1795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:02:50.741114 7559 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:02:50.742236 7559 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:02:50.751628 7564 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:02:50.751641 7565 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:02:50.751838 7559 server_base.cc:1034] running on GCE node
W20260809 02:02:50.751641 7567 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:02:50.752681 7559 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:02:50.754163 7559 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:02:50.755398 7559 hybrid_clock.cc:648] HybridClock initialized: now 1786240970755362 us; error 46 us; skew 500 ppm
I20260809 02:02:50.757926 7559 webserver.cc:460] Webserver started at http://127.1.192.254:44183/ using document root <none> and password file <none>
I20260809 02:02:50.758589 7559 fs_manager.cc:359] Metadata directory not provided
I20260809 02:02:50.758701 7559 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:02:50.758986 7559 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:02:50.761546 7559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786240941968412-1795-0/minicluster-data/master-0/data/instance:
uuid: "2c967a0828374d6ab00abca059a1f4ce"
format_stamp: "Formatted at 2026-08-09 02:02:50 on dist-test-slave-ss4n"
server_key: "87a860fc213d19d1fc8e77feb72282ef"
server_key_iv: "0d96c7b8854da55fd9478f42421d2a6b"
server_key_version: "encryptionkey@0"
I20260809 02:02:50.762415 7559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786240941968412-1795-0/minicluster-data/master-0/wal/instance:
uuid: "2c967a0828374d6ab00abca059a1f4ce"
format_stamp: "Formatted at 2026-08-09 02:02:50 on dist-test-slave-ss4n"
server_key: "87a860fc213d19d1fc8e77feb72282ef"
server_key_iv: "0d96c7b8854da55fd9478f42421d2a6b"
server_key_version: "encryptionkey@0"
I20260809 02:02:50.767136 7559 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 02:02:50.769995 7574 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:50.771327 7559 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 02:02:50.771486 7559 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786240941968412-1795-0/minicluster-data/master-0/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786240941968412-1795-0/minicluster-data/master-0/wal
uuid: "2c967a0828374d6ab00abca059a1f4ce"
format_stamp: "Formatted at 2026-08-09 02:02:50 on dist-test-slave-ss4n"
server_key: "87a860fc213d19d1fc8e77feb72282ef"
server_key_iv: "0d96c7b8854da55fd9478f42421d2a6b"
server_key_version: "encryptionkey@0"
I20260809 02:02:50.771618 7559 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786240941968412-1795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786240941968412-1795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786240941968412-1795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:02:50.788547 7559 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:02:50.789333 7559 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:02:50.789579 7559 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:02:50.796947 7559 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.254:35003
I20260809 02:02:50.796968 7628 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.254:35003 every 8 connection(s)
I20260809 02:02:50.798007 7559 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
I20260809 02:02:50.800779 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 7559
I20260809 02:02:50.800938 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786240941968412-1795-0/minicluster-data/master-0/wal/instance
I20260809 02:02:50.801342 1795 external_mini_cluster.cc:1428] Setting key ad824ad60b1733fbd6a45dd49d08a8c5
I20260809 02:02:50.801898 7629 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:02:50.810935 7629 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c967a0828374d6ab00abca059a1f4ce: Bootstrap starting.
I20260809 02:02:50.813321 7632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:50.803001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35228 (local address 127.1.192.254:35003)
0809 02:02:50.803977 (+ 976us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:50.804002 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:50.804054 (+ 52us) server_negotiation.cc:407] Connection header received
0809 02:02:50.805462 (+ 1408us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:50.805515 (+ 53us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:50.806105 (+ 590us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:50.806537 (+ 432us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:50.807049 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:50.808682 (+ 1633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:50.809625 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:50.810411 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:50.810610 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:50.811495 (+ 885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:02:50.811500 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:02:50.811513 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:02:50.811899 (+ 386us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:02:50.811955 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:02:50.811975 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:02:50.812059 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:50.812265 (+ 206us) server_negotiation.cc:299] Negotiation successful
0809 02:02:50.812575 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":503,"thread_start_us":171,"threads_started":1}
I20260809 02:02:50.814204 7629 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2c967a0828374d6ab00abca059a1f4ce: Neither blocks nor log segments found. Creating new log.
I20260809 02:02:50.815042 7629 log.cc:824] T 00000000000000000000000000000000 P 2c967a0828374d6ab00abca059a1f4ce: Log is configured to *not* fsync() on all Append() calls
I20260809 02:02:50.816406 1795 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 02:02:50.817183 7629 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c967a0828374d6ab00abca059a1f4ce: No bootstrap required, opened a new log
I20260809 02:02:50.820763 7629 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2c967a0828374d6ab00abca059a1f4ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c967a0828374d6ab00abca059a1f4ce" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 35003 } }
I20260809 02:02:50.821031 7629 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2c967a0828374d6ab00abca059a1f4ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:02:50.821115 7629 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2c967a0828374d6ab00abca059a1f4ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c967a0828374d6ab00abca059a1f4ce, State: Initialized, Role: FOLLOWER
I20260809 02:02:50.821601 7629 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2c967a0828374d6ab00abca059a1f4ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c967a0828374d6ab00abca059a1f4ce" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 35003 } }
I20260809 02:02:50.821753 7629 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2c967a0828374d6ab00abca059a1f4ce [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:02:50.821841 7629 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2c967a0828374d6ab00abca059a1f4ce [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:02:50.821940 7629 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2c967a0828374d6ab00abca059a1f4ce [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:02:50.822999 7629 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2c967a0828374d6ab00abca059a1f4ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c967a0828374d6ab00abca059a1f4ce" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 35003 } }
I20260809 02:02:50.823357 7629 leader_election.cc:302] T 00000000000000000000000000000000 P 2c967a0828374d6ab00abca059a1f4ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2c967a0828374d6ab00abca059a1f4ce; no voters:
I20260809 02:02:50.823683 7629 leader_election.cc:288] T 00000000000000000000000000000000 P 2c967a0828374d6ab00abca059a1f4ce [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:02:50.823813 7634 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2c967a0828374d6ab00abca059a1f4ce [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:02:50.824061 7634 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2c967a0828374d6ab00abca059a1f4ce [term 1 LEADER]: Becoming Leader. State: Replica: 2c967a0828374d6ab00abca059a1f4ce, State: Running, Role: LEADER
I20260809 02:02:50.824370 7634 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2c967a0828374d6ab00abca059a1f4ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c967a0828374d6ab00abca059a1f4ce" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 35003 } }
I20260809 02:02:50.824921 7629 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2c967a0828374d6ab00abca059a1f4ce [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:02:50.826135 7637 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c967a0828374d6ab00abca059a1f4ce [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2c967a0828374d6ab00abca059a1f4ce. Latest consensus state: current_term: 1 leader_uuid: "2c967a0828374d6ab00abca059a1f4ce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c967a0828374d6ab00abca059a1f4ce" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 35003 } } }
I20260809 02:02:50.826273 7637 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c967a0828374d6ab00abca059a1f4ce [sys.catalog]: This master's current role is: LEADER
I20260809 02:02:50.826562 7636 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c967a0828374d6ab00abca059a1f4ce [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2c967a0828374d6ab00abca059a1f4ce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c967a0828374d6ab00abca059a1f4ce" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 35003 } } }
I20260809 02:02:50.826666 7636 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c967a0828374d6ab00abca059a1f4ce [sys.catalog]: This master's current role is: LEADER
W20260809 02:02:50.829978 7651 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2c967a0828374d6ab00abca059a1f4ce: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 02:02:50.830058 7651 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 02:02:50.830152 7645 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:02:50.830849 7645 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:02:50.836738 7645 catalog_manager.cc:1347] Generated new cluster ID: 701917cd27ff441381db425371434d69
I20260809 02:02:50.836844 7645 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:02:50.864336 7645 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:02:50.865389 7645 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:02:50.874998 7645 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2c967a0828374d6ab00abca059a1f4ce: Generated new TSK 0
I20260809 02:02:50.875865 7645 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260809 02:02:50.912794 7664 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260809 02:02:50.912914 7664 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260809 02:02:50.913656 7632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:50.895194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35238 (local address 127.1.192.254:35003)
0809 02:02:50.895352 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:50.895358 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:50.895585 (+ 227us) server_negotiation.cc:407] Connection header received
0809 02:02:50.895783 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:50.895787 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:50.895846 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:50.895973 (+ 127us) server_negotiation.cc:226] Negotiated authn=JWT
0809 02:02:50.896277 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:50.897105 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:50.898281 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:50.898623 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:50.898748 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:50.899175 (+ 427us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 02:02:50.913190 (+ 14015us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38815/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0809 02:02:50.913506 (+ 316us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:35238: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38815/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":61}
W20260809 02:02:50.913731 7632 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:35238: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38815/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260809 02:02:50.913930 7663 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:02:50.895073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:35003 (local address 127.0.0.1:35238)
0809 02:02:50.895456 (+ 383us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:50.895476 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 02:02:50.895630 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:02:50.896021 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:02:50.896024 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:02:50.896032 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0809 02:02:50.896158 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:50.896164 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:50.897293 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:50.897296 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:50.898136 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:50.898148 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:50.898808 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:50.898813 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:50.898929 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:50.898966 (+ 37us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 02:02:50.913414 (+ 14448us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38815/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0809 02:02:50.913730 (+ 316us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.1.192.254:35003: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38815/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":144,"threads_started":1}
I20260809 02:02:50.914707 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 7559
2026-08-09T02:02:50Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (353 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260809 02:02:50.956200 1795 mini_oidc.cc:152] Starting JWKS server
I20260809 02:02:50.957242 1795 webserver.cc:460] Webserver started at https://127.0.0.1:42107/ using document root <none> and password file <none>
I20260809 02:02:50.957391 1795 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 02:02:50.957751 1795 webserver.cc:460] Webserver started at http://127.0.0.1:33141/ using document root <none> and password file <none>
2026-08-09T02:02:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:02:50Z Disabled control of system clock
I20260809 02:02:50.968748 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786240941968412-1795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786240941968412-1795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786240941968412-1795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.192.254:41915
--webserver_interface=127.1.192.254
--webserver_port=0
--builtin_ntp_servers=127.1.192.212:44385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.192.254:41915
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42107/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260809 02:02:51.080533 7671 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:02:51.080873 7671 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:02:51.080962 7671 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:02:51.085111 7671 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:02:51.085234 7671 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:02:51.085265 7671 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:02:51.085305 7671 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:02:51.085362 7671 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 02:02:51.085407 7671 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 02:02:51.085440 7671 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:42107/jwks.json
I20260809 02:02:51.090338 7671 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:44385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786240941968412-1795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786240941968412-1795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.192.254:41915
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.192.254:41915
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42107/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.192.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7671
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786240941968412-1795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:02:51.091532 7671 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:02:51.092993 7671 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:02:51.101884 7677 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:02:51.101869 7676 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:02:51.101902 7671 server_base.cc:1034] running on GCE node
W20260809 02:02:51.101950 7679 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:02:51.102679 7671 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:02:51.103834 7671 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:02:51.105034 7671 hybrid_clock.cc:648] HybridClock initialized: now 1786240971104994 us; error 50 us; skew 500 ppm
I20260809 02:02:51.107153 7671 webserver.cc:460] Webserver started at http://127.1.192.254:38161/ using document root <none> and password file <none>
I20260809 02:02:51.107740 7671 fs_manager.cc:359] Metadata directory not provided
I20260809 02:02:51.107831 7671 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:02:51.108073 7671 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:02:51.110306 7671 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786240941968412-1795-0/minicluster-data/master-0/data/instance:
uuid: "ca75e8eb7c754813997f482131c26ab0"
format_stamp: "Formatted at 2026-08-09 02:02:51 on dist-test-slave-ss4n"
server_key: "f3b78703a0c977481c2de2911903c5b5"
server_key_iv: "29905545efa148044f56edd4244495c1"
server_key_version: "encryptionkey@0"
I20260809 02:02:51.110936 7671 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786240941968412-1795-0/minicluster-data/master-0/wal/instance:
uuid: "ca75e8eb7c754813997f482131c26ab0"
format_stamp: "Formatted at 2026-08-09 02:02:51 on dist-test-slave-ss4n"
server_key: "f3b78703a0c977481c2de2911903c5b5"
server_key_iv: "29905545efa148044f56edd4244495c1"
server_key_version: "encryptionkey@0"
I20260809 02:02:51.114874 7671 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 02:02:51.117734 7687 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:51.119133 7671 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 02:02:51.119288 7671 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786240941968412-1795-0/minicluster-data/master-0/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786240941968412-1795-0/minicluster-data/master-0/wal
uuid: "ca75e8eb7c754813997f482131c26ab0"
format_stamp: "Formatted at 2026-08-09 02:02:51 on dist-test-slave-ss4n"
server_key: "f3b78703a0c977481c2de2911903c5b5"
server_key_iv: "29905545efa148044f56edd4244495c1"
server_key_version: "encryptionkey@0"
I20260809 02:02:51.119421 7671 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786240941968412-1795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786240941968412-1795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786240941968412-1795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:02:51.142149 7671 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:02:51.142953 7671 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:02:51.143167 7671 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:02:51.150630 7671 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.254:41915
I20260809 02:02:51.150718 7744 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.254:41915 every 8 connection(s)
I20260809 02:02:51.151654 7671 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
I20260809 02:02:51.155022 7745 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:02:51.155313 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 7671
I20260809 02:02:51.155432 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786240941968412-1795-0/minicluster-data/master-0/wal/instance
I20260809 02:02:51.155755 1795 external_mini_cluster.cc:1428] Setting key d99dad298ae35d623607c8bb3329ef9f
I20260809 02:02:51.161415 7745 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca75e8eb7c754813997f482131c26ab0: Bootstrap starting.
I20260809 02:02:51.163975 7745 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ca75e8eb7c754813997f482131c26ab0: Neither blocks nor log segments found. Creating new log.
I20260809 02:02:51.164788 7745 log.cc:824] T 00000000000000000000000000000000 P ca75e8eb7c754813997f482131c26ab0: Log is configured to *not* fsync() on all Append() calls
I20260809 02:02:51.166622 7748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:51.157312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39726 (local address 127.1.192.254:41915)
0809 02:02:51.158118 (+ 806us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:51.158147 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:51.158190 (+ 43us) server_negotiation.cc:407] Connection header received
0809 02:02:51.159020 (+ 830us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:51.159041 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:51.159483 (+ 442us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:51.159859 (+ 376us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:51.160322 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:51.161393 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:51.162536 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:51.163382 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:51.163590 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:51.164612 (+ 1022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:02:51.164617 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:02:51.164632 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:02:51.165120 (+ 488us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:02:51.165182 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:02:51.165209 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:02:51.165326 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:51.165561 (+ 235us) server_negotiation.cc:299] Negotiation successful
0809 02:02:51.165869 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":364,"thread_start_us":173,"threads_started":1}
I20260809 02:02:51.166832 7745 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca75e8eb7c754813997f482131c26ab0: No bootstrap required, opened a new log
I20260809 02:02:51.169935 1795 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 02:02:51.171160 7745 raft_consensus.cc:348] T 00000000000000000000000000000000 P ca75e8eb7c754813997f482131c26ab0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca75e8eb7c754813997f482131c26ab0" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 41915 } }
I20260809 02:02:51.171459 7745 raft_consensus.cc:374] T 00000000000000000000000000000000 P ca75e8eb7c754813997f482131c26ab0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:02:51.171512 7745 raft_consensus.cc:729] T 00000000000000000000000000000000 P ca75e8eb7c754813997f482131c26ab0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca75e8eb7c754813997f482131c26ab0, State: Initialized, Role: FOLLOWER
I20260809 02:02:51.171933 7745 consensus_queue.cc:260] T 00000000000000000000000000000000 P ca75e8eb7c754813997f482131c26ab0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca75e8eb7c754813997f482131c26ab0" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 41915 } }
I20260809 02:02:51.172075 7745 raft_consensus.cc:388] T 00000000000000000000000000000000 P ca75e8eb7c754813997f482131c26ab0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:02:51.172117 7745 raft_consensus.cc:482] T 00000000000000000000000000000000 P ca75e8eb7c754813997f482131c26ab0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:02:51.172186 7745 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ca75e8eb7c754813997f482131c26ab0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:02:51.173120 7745 raft_consensus.cc:504] T 00000000000000000000000000000000 P ca75e8eb7c754813997f482131c26ab0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca75e8eb7c754813997f482131c26ab0" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 41915 } }
I20260809 02:02:51.173413 7745 leader_election.cc:302] T 00000000000000000000000000000000 P ca75e8eb7c754813997f482131c26ab0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ca75e8eb7c754813997f482131c26ab0; no voters:
I20260809 02:02:51.173748 7745 leader_election.cc:288] T 00000000000000000000000000000000 P ca75e8eb7c754813997f482131c26ab0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:02:51.173938 7750 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ca75e8eb7c754813997f482131c26ab0 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:02:51.174274 7750 raft_consensus.cc:686] T 00000000000000000000000000000000 P ca75e8eb7c754813997f482131c26ab0 [term 1 LEADER]: Becoming Leader. State: Replica: ca75e8eb7c754813997f482131c26ab0, State: Running, Role: LEADER
I20260809 02:02:51.174630 7750 consensus_queue.cc:237] T 00000000000000000000000000000000 P ca75e8eb7c754813997f482131c26ab0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca75e8eb7c754813997f482131c26ab0" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 41915 } }
I20260809 02:02:51.175009 7745 sys_catalog.cc:564] T 00000000000000000000000000000000 P ca75e8eb7c754813997f482131c26ab0 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:02:51.177021 7751 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca75e8eb7c754813997f482131c26ab0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ca75e8eb7c754813997f482131c26ab0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca75e8eb7c754813997f482131c26ab0" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 41915 } } }
I20260809 02:02:51.177196 7751 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca75e8eb7c754813997f482131c26ab0 [sys.catalog]: This master's current role is: LEADER
I20260809 02:02:51.176993 7752 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca75e8eb7c754813997f482131c26ab0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ca75e8eb7c754813997f482131c26ab0. Latest consensus state: current_term: 1 leader_uuid: "ca75e8eb7c754813997f482131c26ab0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca75e8eb7c754813997f482131c26ab0" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 41915 } } }
I20260809 02:02:51.177801 7752 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca75e8eb7c754813997f482131c26ab0 [sys.catalog]: This master's current role is: LEADER
W20260809 02:02:51.180539 7768 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ca75e8eb7c754813997f482131c26ab0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 02:02:51.180640 7768 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 02:02:51.180712 7769 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:02:51.181424 7769 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:02:51.187556 7769 catalog_manager.cc:1347] Generated new cluster ID: 97d6323cce6c43d1a02facde844aa9ae
I20260809 02:02:51.187642 7769 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:02:51.212704 7769 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:02:51.213797 7769 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:02:51.224552 7769 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ca75e8eb7c754813997f482131c26ab0: Generated new TSK 0
I20260809 02:02:51.225358 7769 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:02:51.244473 7748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:51.239084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39738 (local address 127.1.192.254:41915)
0809 02:02:51.239231 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:51.239237 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:51.239384 (+ 147us) server_negotiation.cc:407] Connection header received
0809 02:02:51.239580 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:51.239582 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:51.239632 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:51.239730 (+ 98us) server_negotiation.cc:226] Negotiated authn=JWT
0809 02:02:51.240027 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:51.240591 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:51.241880 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:51.242104 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:51.242205 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:51.242579 (+ 374us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 02:02:51.244064 (+ 1485us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42107/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0809 02:02:51.244315 (+ 251us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:39738: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42107/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":47}
W20260809 02:02:51.244550 7748 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:39738: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42107/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260809 02:02:51.246390 7779 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:02:51.238921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:41915 (local address 127.0.0.1:39738)
0809 02:02:51.239267 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:51.239285 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 02:02:51.239438 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:02:51.239755 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:02:51.239759 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:02:51.239768 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0809 02:02:51.239908 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:51.239916 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:51.240705 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:51.240709 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:51.241751 (+ 1042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:51.241765 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:51.242252 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:51.242257 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:51.242353 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:51.242388 (+ 35us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 02:02:51.244256 (+ 1868us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42107/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0809 02:02:51.246168 (+ 1912us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.1.192.254:41915: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42107/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":251,"mutex_wait_us":1663,"thread_start_us":106,"threads_started":1}
I20260809 02:02:51.247450 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 7671
E20260809 02:02:51.255951 7780 webserver.cc:533] Webserver: sslize failed:
E20260809 02:02:51.256029 7780 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-09T02:02:51Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (333 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-09T02:02:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:02:51Z Disabled control of system clock
I20260809 02:02:51.300207 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786240941968412-1795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786240941968412-1795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786240941968412-1795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.192.254:44859
--webserver_interface=127.1.192.254
--webserver_port=0
--builtin_ntp_servers=127.1.192.212:32835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.192.254:44859
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786240941968412-1795-0/testchainca.cert with env {}
W20260809 02:02:51.421962 7784 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:02:51.422292 7784 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:02:51.422336 7784 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:02:51.426504 7784 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:02:51.426568 7784 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:02:51.426584 7784 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:02:51.426604 7784 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:02:51.426620 7784 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 02:02:51.426637 7784 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 02:02:51.426651 7784 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260809 02:02:51.432179 7784 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:32835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786240941968412-1795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786240941968412-1795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.192.254:44859
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.192.254:44859
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.192.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786240941968412-1795-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7784
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786240941968412-1795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:02:51.433277 7784 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:02:51.434350 7784 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:02:51.443365 7790 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:02:51.443325 7789 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:02:51.443389 7784 server_base.cc:1034] running on GCE node
W20260809 02:02:51.443305 7792 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:02:51.444229 7784 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:02:51.445318 7784 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:02:51.446552 7784 hybrid_clock.cc:648] HybridClock initialized: now 1786240971446528 us; error 57 us; skew 500 ppm
I20260809 02:02:51.448764 7784 webserver.cc:460] Webserver started at http://127.1.192.254:36391/ using document root <none> and password file <none>
I20260809 02:02:51.449425 7784 fs_manager.cc:359] Metadata directory not provided
I20260809 02:02:51.449565 7784 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:02:51.449843 7784 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:02:51.452109 7784 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786240941968412-1795-0/minicluster-data/master-0/data/instance:
uuid: "57406b70ae8547bea59bc5764afc2799"
format_stamp: "Formatted at 2026-08-09 02:02:51 on dist-test-slave-ss4n"
server_key: "d317ed9e0c41ee928df64412f44d3f71"
server_key_iv: "ddf7a7f6f2d830c4651bff6c0742bca1"
server_key_version: "encryptionkey@0"
I20260809 02:02:51.452893 7784 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786240941968412-1795-0/minicluster-data/master-0/wal/instance:
uuid: "57406b70ae8547bea59bc5764afc2799"
format_stamp: "Formatted at 2026-08-09 02:02:51 on dist-test-slave-ss4n"
server_key: "d317ed9e0c41ee928df64412f44d3f71"
server_key_iv: "ddf7a7f6f2d830c4651bff6c0742bca1"
server_key_version: "encryptionkey@0"
I20260809 02:02:51.457165 7784 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.003s
I20260809 02:02:51.459830 7798 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:51.461102 7784 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 02:02:51.461266 7784 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786240941968412-1795-0/minicluster-data/master-0/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786240941968412-1795-0/minicluster-data/master-0/wal
uuid: "57406b70ae8547bea59bc5764afc2799"
format_stamp: "Formatted at 2026-08-09 02:02:51 on dist-test-slave-ss4n"
server_key: "d317ed9e0c41ee928df64412f44d3f71"
server_key_iv: "ddf7a7f6f2d830c4651bff6c0742bca1"
server_key_version: "encryptionkey@0"
I20260809 02:02:51.461395 7784 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786240941968412-1795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786240941968412-1795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786240941968412-1795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:02:51.492340 7784 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:02:51.493196 7784 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:02:51.493425 7784 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:02:51.501765 7784 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.254:44859
I20260809 02:02:51.501784 7858 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.254:44859 every 8 connection(s)
I20260809 02:02:51.502864 7784 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
I20260809 02:02:51.506245 7859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:02:51.507797 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 7784
I20260809 02:02:51.507947 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786240941968412-1795-0/minicluster-data/master-0/wal/instance
I20260809 02:02:51.508257 1795 external_mini_cluster.cc:1428] Setting key f93dc7b4266bc4b8a7dc6e38de67155b
I20260809 02:02:51.513223 7859 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57406b70ae8547bea59bc5764afc2799: Bootstrap starting.
I20260809 02:02:51.516422 7859 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 57406b70ae8547bea59bc5764afc2799: Neither blocks nor log segments found. Creating new log.
I20260809 02:02:51.517390 7859 log.cc:824] T 00000000000000000000000000000000 P 57406b70ae8547bea59bc5764afc2799: Log is configured to *not* fsync() on all Append() calls
I20260809 02:02:51.519279 7862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:51.509864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39980 (local address 127.1.192.254:44859)
0809 02:02:51.510556 (+ 692us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:51.510573 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:51.510620 (+ 47us) server_negotiation.cc:407] Connection header received
0809 02:02:51.511354 (+ 734us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:51.511372 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:51.511808 (+ 436us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:51.512117 (+ 309us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:51.512559 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:51.513940 (+ 1381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:51.515367 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:51.515897 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:51.516037 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:51.517080 (+ 1043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:02:51.517085 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:02:51.517100 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:02:51.517794 (+ 694us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:02:51.517889 (+ 95us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:02:51.517924 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:02:51.518042 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:51.518178 (+ 136us) server_negotiation.cc:299] Negotiation successful
0809 02:02:51.518480 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":163,"threads_started":1}
I20260809 02:02:51.519706 7859 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57406b70ae8547bea59bc5764afc2799: No bootstrap required, opened a new log
I20260809 02:02:51.522413 1795 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 02:02:51.523821 7859 raft_consensus.cc:348] T 00000000000000000000000000000000 P 57406b70ae8547bea59bc5764afc2799 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57406b70ae8547bea59bc5764afc2799" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 44859 } }
I20260809 02:02:51.524106 7859 raft_consensus.cc:374] T 00000000000000000000000000000000 P 57406b70ae8547bea59bc5764afc2799 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:02:51.524209 7859 raft_consensus.cc:729] T 00000000000000000000000000000000 P 57406b70ae8547bea59bc5764afc2799 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57406b70ae8547bea59bc5764afc2799, State: Initialized, Role: FOLLOWER
I20260809 02:02:51.524675 7859 consensus_queue.cc:260] T 00000000000000000000000000000000 P 57406b70ae8547bea59bc5764afc2799 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57406b70ae8547bea59bc5764afc2799" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 44859 } }
I20260809 02:02:51.524808 7859 raft_consensus.cc:388] T 00000000000000000000000000000000 P 57406b70ae8547bea59bc5764afc2799 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:02:51.524863 7859 raft_consensus.cc:482] T 00000000000000000000000000000000 P 57406b70ae8547bea59bc5764afc2799 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:02:51.524937 7859 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 57406b70ae8547bea59bc5764afc2799 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:02:51.525969 7859 raft_consensus.cc:504] T 00000000000000000000000000000000 P 57406b70ae8547bea59bc5764afc2799 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57406b70ae8547bea59bc5764afc2799" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 44859 } }
I20260809 02:02:51.526310 7859 leader_election.cc:302] T 00000000000000000000000000000000 P 57406b70ae8547bea59bc5764afc2799 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 57406b70ae8547bea59bc5764afc2799; no voters:
I20260809 02:02:51.526592 7859 leader_election.cc:288] T 00000000000000000000000000000000 P 57406b70ae8547bea59bc5764afc2799 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:02:51.526760 7864 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 57406b70ae8547bea59bc5764afc2799 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:02:51.527020 7864 raft_consensus.cc:686] T 00000000000000000000000000000000 P 57406b70ae8547bea59bc5764afc2799 [term 1 LEADER]: Becoming Leader. State: Replica: 57406b70ae8547bea59bc5764afc2799, State: Running, Role: LEADER
I20260809 02:02:51.527333 7864 consensus_queue.cc:237] T 00000000000000000000000000000000 P 57406b70ae8547bea59bc5764afc2799 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57406b70ae8547bea59bc5764afc2799" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 44859 } }
I20260809 02:02:51.527872 7859 sys_catalog.cc:564] T 00000000000000000000000000000000 P 57406b70ae8547bea59bc5764afc2799 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:02:51.529121 7866 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57406b70ae8547bea59bc5764afc2799 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "57406b70ae8547bea59bc5764afc2799" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57406b70ae8547bea59bc5764afc2799" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 44859 } } }
I20260809 02:02:51.529265 7866 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57406b70ae8547bea59bc5764afc2799 [sys.catalog]: This master's current role is: LEADER
I20260809 02:02:51.529521 7867 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57406b70ae8547bea59bc5764afc2799 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 57406b70ae8547bea59bc5764afc2799. Latest consensus state: current_term: 1 leader_uuid: "57406b70ae8547bea59bc5764afc2799" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57406b70ae8547bea59bc5764afc2799" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 44859 } } }
I20260809 02:02:51.529623 7867 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57406b70ae8547bea59bc5764afc2799 [sys.catalog]: This master's current role is: LEADER
W20260809 02:02:51.532843 7881 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 57406b70ae8547bea59bc5764afc2799: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 02:02:51.532913 7881 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 02:02:51.533000 7874 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:02:51.533792 7874 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:02:51.539443 7874 catalog_manager.cc:1347] Generated new cluster ID: ab5b00cdda524a71a8db442fbc59a3e7
I20260809 02:02:51.539526 7874 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:02:51.545204 7874 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:02:51.546501 7874 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:02:51.554792 7874 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 57406b70ae8547bea59bc5764afc2799: Generated new TSK 0
I20260809 02:02:51.555662 7874 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:02:51.994547 7862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:51.565587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39988 (local address 127.1.192.254:44859)
0809 02:02:51.565729 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:51.565734 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:51.565895 (+ 161us) server_negotiation.cc:407] Connection header received
0809 02:02:51.566086 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:51.566090 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:51.566148 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:51.566276 (+ 128us) server_negotiation.cc:226] Negotiated authn=JWT
0809 02:02:51.566536 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:51.567403 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:51.568296 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:51.568654 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:51.568762 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:51.569052 (+ 290us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 02:02:51.994091 (+425039us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0809 02:02:51.994394 (+ 303us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:39988: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":52}
W20260809 02:02:51.994692 7862 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:39988: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260809 02:02:51.994700 7889 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:02:51.565435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:44859 (local address 127.0.0.1:39988)
0809 02:02:51.565806 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:51.565819 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:02:51.565922 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:02:51.566298 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:02:51.566302 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:02:51.566313 (+ 11us) client_negotiation.cc:189] Negotiated authn=JWT
0809 02:02:51.566425 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:51.566430 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:51.567550 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:51.567552 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:51.568185 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:51.568193 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:51.568780 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:51.568783 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:51.568848 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:51.568871 (+ 23us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 02:02:51.994346 (+425475us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0809 02:02:51.994536 (+ 190us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.1.192.254:44859: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":124,"threads_started":1}
I20260809 02:02:51.995595 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 7784
2026-08-09T02:02:51Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (728 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (140 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (129 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786240941968412-1795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:02:52 dist-test-slave-ss4n krb5kdc[7902](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:02:52 dist-test-slave-ss4n krb5kdc[7902](info): set up 2 sockets
Aug 09 02:02:52 dist-test-slave-ss4n krb5kdc[7902](info): commencing operation
krb5kdc: starting...
W20260809 02:02:53.419824 1795 mini_kdc.cc:121] Time spent starting KDC: real 1.132s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786240941968412-1795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786240941968412-1795-0/krb5kdc/test-user.keytab.
Aug 09 02:02:53 dist-test-slave-ss4n krb5kdc[7902](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240973, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:02:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:02:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.192.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.192.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.192.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab.
Entry for principal kudu/127.1.192.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab.
I20260809 02:02:53.558982 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786240941968412-1795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786240941968412-1795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786240941968412-1795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.192.254:40687
--webserver_interface=127.1.192.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab
--principal=kudu/127.1.192.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:33649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.192.254:40687
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:02:53.702319 7917 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:02:53.702675 7917 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:02:53.702760 7917 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:02:53.706898 7917 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:02:53.707011 7917 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:02:53.707051 7917 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:02:53.707091 7917 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:02:53.707126 7917 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:02:53.712817 7917 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:33649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786240941968412-1795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786240941968412-1795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.192.254:40687
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab
--principal=kudu/127.1.192.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.192.254:40687
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.192.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7917
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786240941968412-1795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:02:53.714059 7917 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:02:53.715167 7917 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:02:53.723999 7925 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:02:53.723982 7927 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:02:53.724035 7917 server_base.cc:1034] running on GCE node
W20260809 02:02:53.723982 7924 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:02:53.724869 7917 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:02:53.726197 7917 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:02:53.727391 7917 hybrid_clock.cc:648] HybridClock initialized: now 1786240973727374 us; error 40 us; skew 500 ppm
Aug 09 02:02:53 dist-test-slave-ss4n krb5kdc[7902](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240973, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:02:53.733320 7917 init.cc:377] Logged in from keytab as kudu/127.1.192.254@KRBTEST.COM (short username kudu)
I20260809 02:02:53.734328 7917 webserver.cc:460] Webserver started at http://127.1.192.254:43841/ using document root <none> and password file <none>
I20260809 02:02:53.734954 7917 fs_manager.cc:359] Metadata directory not provided
I20260809 02:02:53.735057 7917 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:02:53.735318 7917 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:02:53.737561 7917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786240941968412-1795-0/minicluster-data/master-0/data/instance:
uuid: "6744905950664c0d934a46c143b7a1f1"
format_stamp: "Formatted at 2026-08-09 02:02:53 on dist-test-slave-ss4n"
server_key: "ea38f0f9e5b3fe95b174ee2856818a88"
server_key_iv: "a3095575de6a6ba088d8a2c54d8da597"
server_key_version: "encryptionkey@0"
I20260809 02:02:53.738181 7917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786240941968412-1795-0/minicluster-data/master-0/wal/instance:
uuid: "6744905950664c0d934a46c143b7a1f1"
format_stamp: "Formatted at 2026-08-09 02:02:53 on dist-test-slave-ss4n"
server_key: "ea38f0f9e5b3fe95b174ee2856818a88"
server_key_iv: "a3095575de6a6ba088d8a2c54d8da597"
server_key_version: "encryptionkey@0"
I20260809 02:02:53.742203 7917 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260809 02:02:53.744810 7934 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:53.746199 7917 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 02:02:53.746353 7917 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786240941968412-1795-0/minicluster-data/master-0/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786240941968412-1795-0/minicluster-data/master-0/wal
uuid: "6744905950664c0d934a46c143b7a1f1"
format_stamp: "Formatted at 2026-08-09 02:02:53 on dist-test-slave-ss4n"
server_key: "ea38f0f9e5b3fe95b174ee2856818a88"
server_key_iv: "a3095575de6a6ba088d8a2c54d8da597"
server_key_version: "encryptionkey@0"
I20260809 02:02:53.746513 7917 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786240941968412-1795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786240941968412-1795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786240941968412-1795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:02:53.768512 7917 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:02:53.771835 7917 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:02:53.772054 7917 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:02:53.779680 7917 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.254:40687
I20260809 02:02:53.779703 7992 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.254:40687 every 8 connection(s)
I20260809 02:02:53.780685 7917 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
I20260809 02:02:53.783548 7993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:02:53.787358 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 7917
I20260809 02:02:53.787492 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786240941968412-1795-0/minicluster-data/master-0/wal/instance
I20260809 02:02:53.787747 1795 external_mini_cluster.cc:1428] Setting key c012dad3cf99d4bf9b5ec4027caba0a2
I20260809 02:02:53.789911 7993 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6744905950664c0d934a46c143b7a1f1: Bootstrap starting.
I20260809 02:02:53.792740 7993 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6744905950664c0d934a46c143b7a1f1: Neither blocks nor log segments found. Creating new log.
I20260809 02:02:53.793686 7993 log.cc:824] T 00000000000000000000000000000000 P 6744905950664c0d934a46c143b7a1f1: Log is configured to *not* fsync() on all Append() calls
I20260809 02:02:53.796092 7993 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6744905950664c0d934a46c143b7a1f1: No bootstrap required, opened a new log
Aug 09 02:02:53 dist-test-slave-ss4n krb5kdc[7902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240973, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.192.254@KRBTEST.COM
I20260809 02:02:53.800632 7993 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6744905950664c0d934a46c143b7a1f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6744905950664c0d934a46c143b7a1f1" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 40687 } }
I20260809 02:02:53.800997 7993 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6744905950664c0d934a46c143b7a1f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:02:53.801067 7993 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6744905950664c0d934a46c143b7a1f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6744905950664c0d934a46c143b7a1f1, State: Initialized, Role: FOLLOWER
I20260809 02:02:53.801633 7993 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6744905950664c0d934a46c143b7a1f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6744905950664c0d934a46c143b7a1f1" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 40687 } }
I20260809 02:02:53.801832 7993 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6744905950664c0d934a46c143b7a1f1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:02:53.801940 7993 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6744905950664c0d934a46c143b7a1f1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:02:53.802062 7993 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6744905950664c0d934a46c143b7a1f1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:02:53.803383 7993 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6744905950664c0d934a46c143b7a1f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6744905950664c0d934a46c143b7a1f1" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 40687 } }
I20260809 02:02:53.803755 7993 leader_election.cc:302] T 00000000000000000000000000000000 P 6744905950664c0d934a46c143b7a1f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6744905950664c0d934a46c143b7a1f1; no voters:
I20260809 02:02:53.804112 7993 leader_election.cc:288] T 00000000000000000000000000000000 P 6744905950664c0d934a46c143b7a1f1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:02:53.804224 7997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:53.789257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32984 (local address 127.1.192.254:40687)
0809 02:02:53.789726 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:53.789746 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:53.789791 (+ 45us) server_negotiation.cc:407] Connection header received
0809 02:02:53.790619 (+ 828us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:53.790643 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:53.791094 (+ 451us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:53.791499 (+ 405us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:53.792274 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:53.793831 (+ 1557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:53.794897 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:53.795586 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:53.795756 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:53.798693 (+ 2937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:02:53.798699 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:02:53.798715 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:02:53.798747 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:02:53.800608 (+ 1861us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:53.801243 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:53.801248 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:53.801253 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:53.801425 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:53.801837 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:53.801840 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:53.801842 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:53.802417 (+ 575us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:02:53.802589 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:53.802909 (+ 320us) server_negotiation.cc:299] Negotiation successful
0809 02:02:53.803250 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":192,"threads_started":1}
I20260809 02:02:53.804207 7999 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6744905950664c0d934a46c143b7a1f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:02:53.804461 7999 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6744905950664c0d934a46c143b7a1f1 [term 1 LEADER]: Becoming Leader. State: Replica: 6744905950664c0d934a46c143b7a1f1, State: Running, Role: LEADER
I20260809 02:02:53.804788 7999 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6744905950664c0d934a46c143b7a1f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6744905950664c0d934a46c143b7a1f1" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 40687 } }
I20260809 02:02:53.805452 7993 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6744905950664c0d934a46c143b7a1f1 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:02:53.806078 7999 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6744905950664c0d934a46c143b7a1f1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6744905950664c0d934a46c143b7a1f1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6744905950664c0d934a46c143b7a1f1" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 40687 } } }
I20260809 02:02:53.806290 7999 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6744905950664c0d934a46c143b7a1f1 [sys.catalog]: This master's current role is: LEADER
I20260809 02:02:53.806577 7999 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6744905950664c0d934a46c143b7a1f1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6744905950664c0d934a46c143b7a1f1. Latest consensus state: current_term: 1 leader_uuid: "6744905950664c0d934a46c143b7a1f1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6744905950664c0d934a46c143b7a1f1" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 40687 } } }
I20260809 02:02:53.806718 7999 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6744905950664c0d934a46c143b7a1f1 [sys.catalog]: This master's current role is: LEADER
I20260809 02:02:53.807037 8007 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:02:53.809932 8007 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:02:53.815796 8007 catalog_manager.cc:1347] Generated new cluster ID: af405dc0e08749bc9cc2a54f515d4392
I20260809 02:02:53.815937 8007 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:02:53.827582 8007 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:02:53.828835 8007 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:02:53.839530 8007 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6744905950664c0d934a46c143b7a1f1: Generated new TSK 0
I20260809 02:02:53.840445 8007 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.192.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.192.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.192.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.193.keytab.
Entry for principal kudu/127.1.192.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.193.keytab.
I20260809 02:02:53.882125 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786240941968412-1795-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786240941968412-1795-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786240941968412-1795-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786240941968412-1795-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.192.193:0
--local_ip_for_outbound_sockets=127.1.192.193
--webserver_interface=127.1.192.193
--webserver_port=0
--tserver_master_addrs=127.1.192.254:40687
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.193.keytab
--principal=kudu/127.1.192.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:33649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:02:54.007248 8023 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:02:54.007616 8023 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:02:54.007710 8023 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:02:54.011987 8023 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:02:54.012091 8023 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:02:54.012188 8023 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.192.193
I20260809 02:02:54.017699 8023 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:33649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786240941968412-1795-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786240941968412-1795-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.193.keytab
--principal=kudu/127.1.192.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.192.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786240941968412-1795-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.192.193
--webserver_port=0
--tserver_master_addrs=127.1.192.254:40687
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8023
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.192.193
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786240941968412-1795-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:02:54.018852 8023 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:02:54.020027 8023 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:02:54.030009 8028 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:02:54.030023 8029 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:02:54.030023 8031 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:02:54.030344 8023 server_base.cc:1034] running on GCE node
I20260809 02:02:54.030825 8023 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:02:54.031497 8023 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:02:54.032735 8023 hybrid_clock.cc:648] HybridClock initialized: now 1786240974032711 us; error 35 us; skew 500 ppm
Aug 09 02:02:54 dist-test-slave-ss4n krb5kdc[7902](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240974, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:02:54.035989 8023 init.cc:377] Logged in from keytab as kudu/127.1.192.193@KRBTEST.COM (short username kudu)
I20260809 02:02:54.037050 8023 webserver.cc:460] Webserver started at http://127.1.192.193:33227/ using document root <none> and password file <none>
I20260809 02:02:54.037720 8023 fs_manager.cc:359] Metadata directory not provided
I20260809 02:02:54.037820 8023 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:02:54.038079 8023 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:02:54.040289 8023 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786240941968412-1795-0/minicluster-data/ts-0/data/instance:
uuid: "84cf90213004483e921a997a1d06aeec"
format_stamp: "Formatted at 2026-08-09 02:02:54 on dist-test-slave-ss4n"
server_key: "da81eda7e796c0f07b90772eea8ad123"
server_key_iv: "c02f6286cfaa4ec41adc7f16d4e11977"
server_key_version: "encryptionkey@0"
I20260809 02:02:54.040915 8023 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786240941968412-1795-0/minicluster-data/ts-0/wal/instance:
uuid: "84cf90213004483e921a997a1d06aeec"
format_stamp: "Formatted at 2026-08-09 02:02:54 on dist-test-slave-ss4n"
server_key: "da81eda7e796c0f07b90772eea8ad123"
server_key_iv: "c02f6286cfaa4ec41adc7f16d4e11977"
server_key_version: "encryptionkey@0"
I20260809 02:02:54.045720 8023 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260809 02:02:54.048379 8038 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:54.049820 8023 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 02:02:54.049961 8023 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786240941968412-1795-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786240941968412-1795-0/minicluster-data/ts-0/wal
uuid: "84cf90213004483e921a997a1d06aeec"
format_stamp: "Formatted at 2026-08-09 02:02:54 on dist-test-slave-ss4n"
server_key: "da81eda7e796c0f07b90772eea8ad123"
server_key_iv: "c02f6286cfaa4ec41adc7f16d4e11977"
server_key_version: "encryptionkey@0"
I20260809 02:02:54.050084 8023 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786240941968412-1795-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786240941968412-1795-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786240941968412-1795-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:02:54.087833 8023 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:02:54.091111 8023 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:02:54.091343 8023 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:02:54.091989 8023 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:02:54.092876 8023 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:02:54.092952 8023 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:54.093029 8023 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:02:54.093073 8023 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:54.102442 8023 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.193:34417
I20260809 02:02:54.102468 8164 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.193:34417 every 8 connection(s)
I20260809 02:02:54.103338 8023 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786240941968412-1795-0/minicluster-data/ts-0/data/info.pb
I20260809 02:02:54.111433 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 8023
I20260809 02:02:54.111568 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786240941968412-1795-0/minicluster-data/ts-0/wal/instance
I20260809 02:02:54.111958 1795 external_mini_cluster.cc:1428] Setting key f0abc78dcdbceada51ba5d04c0a0fb09
Aug 09 02:02:54 dist-test-slave-ss4n krb5kdc[7902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240974, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.193@KRBTEST.COM for kudu/127.1.192.254@KRBTEST.COM
I20260809 02:02:54.118497 7997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:54.105219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.193:55909 (local address 127.1.192.254:40687)
0809 02:02:54.105404 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:54.105409 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:54.106317 (+ 908us) server_negotiation.cc:407] Connection header received
0809 02:02:54.107883 (+ 1566us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:54.107886 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:54.107936 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:54.108013 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:54.109213 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:54.109823 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:54.110938 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:54.111162 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:54.111242 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:54.114248 (+ 3006us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:02:54.114255 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:02:54.114261 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:02:54.114295 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:02:54.116051 (+ 1756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:54.116583 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:54.116591 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:54.116593 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:54.116681 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:54.117038 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:54.117043 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:54.117044 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:54.117275 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:02:54.117409 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:54.118107 (+ 698us) server_negotiation.cc:299] Negotiation successful
0809 02:02:54.118274 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260809 02:02:54.119733 8167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:54.105625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:40687 (local address 127.1.192.193:55909)
0809 02:02:54.106162 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:54.106206 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 02:02:54.107686 (+ 1480us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:02:54.108179 (+ 493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:02:54.108185 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:02:54.108545 (+ 360us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:02:54.109067 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:54.109080 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:54.109938 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:54.109940 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:54.110828 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:54.110837 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:54.111338 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:54.111343 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:54.111563 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:54.112271 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:02:54.112287 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:02:54.114068 (+ 1781us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:02:54.116177 (+ 2109us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:54.116183 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:54.116198 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:54.116465 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:54.116779 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:54.116782 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:54.116784 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:54.116936 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:54.117406 (+ 470us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:02:54.117413 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:02:54.117790 (+ 377us) client_negotiation.cc:769] Sending connection context
0809 02:02:54.118045 (+ 255us) client_negotiation.cc:240] Negotiation successful
0809 02:02:54.118456 (+ 411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":171,"threads_started":1}
I20260809 02:02:54.121294 8165 heartbeater.cc:347] Connected to a master server at 127.1.192.254:40687
I20260809 02:02:54.121618 8165 heartbeater.cc:464] Registering TS with master...
I20260809 02:02:54.122283 8165 heartbeater.cc:511] Master 127.1.192.254:40687 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.1.192.194@KRBTEST.COM; defaulting to no policy
I20260809 02:02:54.123987 7953 ts_manager.cc:194] Registered new tserver with Master: 84cf90213004483e921a997a1d06aeec (127.1.192.193:34417)
I20260809 02:02:54.125116 7953 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.192.193@KRBTEST.COM'} at 127.1.192.193:55909
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.192.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.192.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.194.keytab.
Entry for principal kudu/127.1.192.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.194.keytab.
I20260809 02:02:54.145797 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786240941968412-1795-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786240941968412-1795-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786240941968412-1795-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786240941968412-1795-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.192.194:0
--local_ip_for_outbound_sockets=127.1.192.194
--webserver_interface=127.1.192.194
--webserver_port=0
--tserver_master_addrs=127.1.192.254:40687
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.194.keytab
--principal=kudu/127.1.192.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:33649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:02:54.259730 8170 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:02:54.260017 8170 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:02:54.260092 8170 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:02:54.264256 8170 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:02:54.264328 8170 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:02:54.264424 8170 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.192.194
I20260809 02:02:54.269951 8170 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:33649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786240941968412-1795-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786240941968412-1795-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.194.keytab
--principal=kudu/127.1.192.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.192.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786240941968412-1795-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.192.194
--webserver_port=0
--tserver_master_addrs=127.1.192.254:40687
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8170
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.192.194
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786240941968412-1795-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:02:54.271085 8170 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:02:54.272162 8170 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:02:54.282097 8178 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:02:54.282104 8175 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:02:54.282267 8176 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:02:54.282413 8170 server_base.cc:1034] running on GCE node
I20260809 02:02:54.282955 8170 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:02:54.283640 8170 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:02:54.284842 8170 hybrid_clock.cc:648] HybridClock initialized: now 1786240974284814 us; error 39 us; skew 500 ppm
Aug 09 02:02:54 dist-test-slave-ss4n krb5kdc[7902](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240974, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:02:54.287822 8170 init.cc:377] Logged in from keytab as kudu/127.1.192.194@KRBTEST.COM (short username kudu)
I20260809 02:02:54.288779 8170 webserver.cc:460] Webserver started at http://127.1.192.194:44669/ using document root <none> and password file <none>
I20260809 02:02:54.289353 8170 fs_manager.cc:359] Metadata directory not provided
I20260809 02:02:54.289450 8170 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:02:54.289728 8170 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:02:54.291810 8170 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786240941968412-1795-0/minicluster-data/ts-1/data/instance:
uuid: "2bb0cdca112549b8ae67170605485011"
format_stamp: "Formatted at 2026-08-09 02:02:54 on dist-test-slave-ss4n"
server_key: "92e1849b06c053c0d81e78b643a33561"
server_key_iv: "b22021fec35314b76169b3a345aff403"
server_key_version: "encryptionkey@0"
I20260809 02:02:54.292384 8170 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786240941968412-1795-0/minicluster-data/ts-1/wal/instance:
uuid: "2bb0cdca112549b8ae67170605485011"
format_stamp: "Formatted at 2026-08-09 02:02:54 on dist-test-slave-ss4n"
server_key: "92e1849b06c053c0d81e78b643a33561"
server_key_iv: "b22021fec35314b76169b3a345aff403"
server_key_version: "encryptionkey@0"
I20260809 02:02:54.296228 8170 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 02:02:54.298841 8186 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:54.300077 8170 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 02:02:54.300220 8170 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786240941968412-1795-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786240941968412-1795-0/minicluster-data/ts-1/wal
uuid: "2bb0cdca112549b8ae67170605485011"
format_stamp: "Formatted at 2026-08-09 02:02:54 on dist-test-slave-ss4n"
server_key: "92e1849b06c053c0d81e78b643a33561"
server_key_iv: "b22021fec35314b76169b3a345aff403"
server_key_version: "encryptionkey@0"
I20260809 02:02:54.300338 8170 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786240941968412-1795-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786240941968412-1795-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786240941968412-1795-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:02:54.310482 8170 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:02:54.314157 8170 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:02:54.314375 8170 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:02:54.315009 8170 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:02:54.315857 8170 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:02:54.315941 8170 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:54.316033 8170 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:02:54.316087 8170 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:54.325731 8170 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.194:37053
I20260809 02:02:54.325764 8309 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.194:37053 every 8 connection(s)
I20260809 02:02:54.326690 8170 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786240941968412-1795-0/minicluster-data/ts-1/data/info.pb
I20260809 02:02:54.333663 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 8170
I20260809 02:02:54.333798 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786240941968412-1795-0/minicluster-data/ts-1/wal/instance
I20260809 02:02:54.334087 1795 external_mini_cluster.cc:1428] Setting key b8cbaeb12cea79eaf234529c69891f4b
Aug 09 02:02:54 dist-test-slave-ss4n krb5kdc[7902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240974, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.194@KRBTEST.COM for kudu/127.1.192.254@KRBTEST.COM
I20260809 02:02:54.343194 7997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:54.328623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.194:43499 (local address 127.1.192.254:40687)
0809 02:02:54.328813 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:54.328818 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:54.329850 (+ 1032us) server_negotiation.cc:407] Connection header received
0809 02:02:54.331240 (+ 1390us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:54.331243 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:54.331293 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:54.331377 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:54.332567 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:54.333122 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:54.334791 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:54.335150 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:54.335243 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:54.338424 (+ 3181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:02:54.338427 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:02:54.338429 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:02:54.338458 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:02:54.340231 (+ 1773us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:54.341019 (+ 788us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:54.341022 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:54.341023 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:54.341111 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:54.341688 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:54.341690 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:54.341692 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:54.341916 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:02:54.342007 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:54.342898 (+ 891us) server_negotiation.cc:299] Negotiation successful
0809 02:02:54.343054 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260809 02:02:54.344600 8312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:54.329086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:40687 (local address 127.1.192.194:43499)
0809 02:02:54.329687 (+ 601us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:54.329739 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 02:02:54.330997 (+ 1258us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:02:54.331534 (+ 537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:02:54.331540 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:02:54.331945 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:02:54.332422 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:54.332435 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:54.333235 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:54.333238 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:54.334612 (+ 1374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:54.334621 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:54.335314 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:54.335318 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:54.335505 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:54.336198 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:02:54.336219 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:02:54.338214 (+ 1995us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:02:54.340375 (+ 2161us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:54.340385 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:54.340408 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:54.340881 (+ 473us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:54.341255 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:54.341259 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:54.341262 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:54.341538 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:54.342044 (+ 506us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:02:54.342056 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:02:54.342515 (+ 459us) client_negotiation.cc:769] Sending connection context
0809 02:02:54.342791 (+ 276us) client_negotiation.cc:240] Negotiation successful
0809 02:02:54.343252 (+ 461us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":140,"threads_started":1}
I20260809 02:02:54.346143 8310 heartbeater.cc:347] Connected to a master server at 127.1.192.254:40687
I20260809 02:02:54.346436 8310 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.1.192.195@KRBTEST.COM; defaulting to no policy
I20260809 02:02:54.347097 8310 heartbeater.cc:511] Master 127.1.192.254:40687 requested a full tablet report, sending...
I20260809 02:02:54.348407 7953 ts_manager.cc:194] Registered new tserver with Master: 2bb0cdca112549b8ae67170605485011 (127.1.192.194:37053)
I20260809 02:02:54.349166 7953 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.192.194@KRBTEST.COM'} at 127.1.192.194:43499
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.192.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.192.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.195.keytab.
Entry for principal kudu/127.1.192.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.195.keytab.
I20260809 02:02:54.369858 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786240941968412-1795-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786240941968412-1795-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786240941968412-1795-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786240941968412-1795-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.192.195:0
--local_ip_for_outbound_sockets=127.1.192.195
--webserver_interface=127.1.192.195
--webserver_port=0
--tserver_master_addrs=127.1.192.254:40687
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.195.keytab
--principal=kudu/127.1.192.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:33649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:02:54.486712 8315 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:02:54.487013 8315 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:02:54.487094 8315 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:02:54.491212 8315 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:02:54.491303 8315 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:02:54.491392 8315 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.192.195
I20260809 02:02:54.496814 8315 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:33649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786240941968412-1795-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786240941968412-1795-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.195.keytab
--principal=kudu/127.1.192.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.192.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786240941968412-1795-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.192.195
--webserver_port=0
--tserver_master_addrs=127.1.192.254:40687
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8315
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.192.195
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786240941968412-1795-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:02:54.498205 8315 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:02:54.499420 8315 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:02:54.509614 8320 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:02:54.509651 8321 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:02:54.509708 8323 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:02:54.510160 8315 server_base.cc:1034] running on GCE node
I20260809 02:02:54.510592 8315 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:02:54.511301 8315 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:02:54.512506 8315 hybrid_clock.cc:648] HybridClock initialized: now 1786240974512494 us; error 38 us; skew 500 ppm
Aug 09 02:02:54 dist-test-slave-ss4n krb5kdc[7902](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240974, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:02:54.515588 8315 init.cc:377] Logged in from keytab as kudu/127.1.192.195@KRBTEST.COM (short username kudu)
I20260809 02:02:54.516544 8315 webserver.cc:460] Webserver started at http://127.1.192.195:36083/ using document root <none> and password file <none>
I20260809 02:02:54.517230 8315 fs_manager.cc:359] Metadata directory not provided
I20260809 02:02:54.517302 8315 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:02:54.517540 8315 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:02:54.519685 8315 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786240941968412-1795-0/minicluster-data/ts-2/data/instance:
uuid: "c1e3d3bd5fde4c0eabaa16444989d35b"
format_stamp: "Formatted at 2026-08-09 02:02:54 on dist-test-slave-ss4n"
server_key: "88ccb3e5d6f360bba29322fad79e1e3e"
server_key_iv: "c0b027ed3c14f52c9468ccd8ca993de9"
server_key_version: "encryptionkey@0"
I20260809 02:02:54.520263 8315 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786240941968412-1795-0/minicluster-data/ts-2/wal/instance:
uuid: "c1e3d3bd5fde4c0eabaa16444989d35b"
format_stamp: "Formatted at 2026-08-09 02:02:54 on dist-test-slave-ss4n"
server_key: "88ccb3e5d6f360bba29322fad79e1e3e"
server_key_iv: "c0b027ed3c14f52c9468ccd8ca993de9"
server_key_version: "encryptionkey@0"
I20260809 02:02:54.524416 8315 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260809 02:02:54.527283 8330 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:54.528775 8315 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 02:02:54.528920 8315 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786240941968412-1795-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786240941968412-1795-0/minicluster-data/ts-2/wal
uuid: "c1e3d3bd5fde4c0eabaa16444989d35b"
format_stamp: "Formatted at 2026-08-09 02:02:54 on dist-test-slave-ss4n"
server_key: "88ccb3e5d6f360bba29322fad79e1e3e"
server_key_iv: "c0b027ed3c14f52c9468ccd8ca993de9"
server_key_version: "encryptionkey@0"
I20260809 02:02:54.529050 8315 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786240941968412-1795-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786240941968412-1795-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786240941968412-1795-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:02:54.560178 8315 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:02:54.563868 8315 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:02:54.564047 8315 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:02:54.564775 8315 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:02:54.565698 8315 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:02:54.565748 8315 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:54.565807 8315 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:02:54.565840 8315 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:54.575363 8315 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.195:43335
I20260809 02:02:54.575387 8456 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.195:43335 every 8 connection(s)
I20260809 02:02:54.576329 8315 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786240941968412-1795-0/minicluster-data/ts-2/data/info.pb
I20260809 02:02:54.578255 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 8315
I20260809 02:02:54.578434 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786240941968412-1795-0/minicluster-data/ts-2/wal/instance
I20260809 02:02:54.578778 1795 external_mini_cluster.cc:1428] Setting key a2e699cffcd94a9188b908d0fdb43414
Aug 09 02:02:54 dist-test-slave-ss4n krb5kdc[7902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240974, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.195@KRBTEST.COM for kudu/127.1.192.254@KRBTEST.COM
I20260809 02:02:54.590768 7997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:54.578445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.195:57557 (local address 127.1.192.254:40687)
0809 02:02:54.578619 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:54.578626 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:54.579627 (+ 1001us) server_negotiation.cc:407] Connection header received
0809 02:02:54.581025 (+ 1398us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:54.581028 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:54.581083 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:54.581197 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:54.582425 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:54.582976 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:54.584142 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:54.584369 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:54.584486 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:54.587177 (+ 2691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:02:54.587180 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:02:54.587182 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:02:54.587208 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:02:54.588719 (+ 1511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:54.589222 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:54.589224 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:54.589225 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:54.589300 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:54.589712 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:54.589714 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:54.589715 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:54.589895 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:02:54.589983 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:54.590501 (+ 518us) server_negotiation.cc:299] Negotiation successful
0809 02:02:54.590638 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 02:02:54.591815 8460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:54.578911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:40687 (local address 127.1.192.195:57557)
0809 02:02:54.579427 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:54.579482 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0809 02:02:54.580809 (+ 1327us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:02:54.581375 (+ 566us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:02:54.581382 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:02:54.581771 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:02:54.582271 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:54.582285 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:54.583128 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:54.583131 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:54.583999 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:54.584008 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:54.584526 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:54.584530 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:54.584711 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:54.585356 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:02:54.585372 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:02:54.587003 (+ 1631us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:02:54.588825 (+ 1822us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:54.588830 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:54.588843 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:54.589124 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:54.589388 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:54.589390 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:54.589392 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:54.589610 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:54.590002 (+ 392us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:02:54.590008 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:02:54.590275 (+ 267us) client_negotiation.cc:769] Sending connection context
0809 02:02:54.590441 (+ 166us) client_negotiation.cc:240] Negotiation successful
0809 02:02:54.590742 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":162,"threads_started":1}
I20260809 02:02:54.592950 8458 heartbeater.cc:347] Connected to a master server at 127.1.192.254:40687
I20260809 02:02:54.593238 8458 heartbeater.cc:464] Registering TS with master...
I20260809 02:02:54.593828 8458 heartbeater.cc:511] Master 127.1.192.254:40687 requested a full tablet report, sending...
I20260809 02:02:54.594854 7953 ts_manager.cc:194] Registered new tserver with Master: c1e3d3bd5fde4c0eabaa16444989d35b (127.1.192.195:43335)
I20260809 02:02:54.595448 7953 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.192.195@KRBTEST.COM'} at 127.1.192.195:57557
I20260809 02:02:54.605211 1795 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786240941968412-1795-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786240941968412-1795-0/krb5kdc/test-admin.keytab.
Aug 09 02:02:54 dist-test-slave-ss4n krb5kdc[7902](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240974, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:02:54.624334 1795 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260809 02:02:54.627434 8465 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:02:54.625937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42036 (local address 127.1.192.193:34417)
0809 02:02:54.626278 (+ 341us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:54.626284 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:54.626308 (+ 24us) server_negotiation.cc:407] Connection header received
0809 02:02:54.626365 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:54.626370 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:54.626555 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:54.626691 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:54.627282 (+ 591us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42036: BlockingRecv error: recv got EOF from 127.0.0.1:42036 (error 108)
Metrics: {"server-negotiator.queue_time_us":240,"thread_start_us":142,"threads_started":1}
W20260809 02:02:54.627607 1795 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786240941968412-1795-0/krb5kdc/krb5cc' of type FILE
Aug 09 02:02:54 dist-test-slave-ss4n krb5kdc[7902](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240974, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:02:54.628818 1795 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 02:02:54 dist-test-slave-ss4n krb5kdc[7902](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240974, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:02:54.630159 1795 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 02:02:54 dist-test-slave-ss4n krb5kdc[7902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240974, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.192.193@KRBTEST.COM
I20260809 02:02:54.641654 8465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:54.631177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42038 (local address 127.1.192.193:34417)
0809 02:02:54.631343 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:54.631347 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:54.631426 (+ 79us) server_negotiation.cc:407] Connection header received
0809 02:02:54.631533 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:54.631535 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:54.631576 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:54.631650 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:54.632236 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:54.633021 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:54.634096 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:54.634481 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:54.634570 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:54.637176 (+ 2606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:02:54.637188 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:02:54.637202 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:02:54.637229 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:02:54.639055 (+ 1826us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:54.639869 (+ 814us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:54.639876 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:54.639877 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:54.639983 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:54.640348 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:54.640354 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:54.640356 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:54.640728 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:02:54.641038 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:54.641256 (+ 218us) server_negotiation.cc:299] Negotiation successful
0809 02:02:54.641446 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
W20260809 02:02:54.646178 8465 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:02:54.644878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42054 (local address 127.1.192.193:34417)
0809 02:02:54.645019 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:54.645023 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:54.645163 (+ 140us) server_negotiation.cc:407] Connection header received
0809 02:02:54.645312 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:54.645315 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:54.645364 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:54.645452 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:54.646066 (+ 614us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42054: BlockingRecv error: recv got EOF from 127.0.0.1:42054 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260809 02:02:54.646339 1795 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786240941968412-1795-0/krb5kdc/krb5cc' of type FILE
Aug 09 02:02:54 dist-test-slave-ss4n krb5kdc[7902](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240974, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:02:54.647454 1795 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 02:02:54 dist-test-slave-ss4n krb5kdc[7902](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240974, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:02:54.648499 1795 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 02:02:54 dist-test-slave-ss4n krb5kdc[7902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240974, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.192.193@KRBTEST.COM
I20260809 02:02:54.659098 8465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:54.649455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42056 (local address 127.1.192.193:34417)
0809 02:02:54.649620 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:54.649624 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:54.649677 (+ 53us) server_negotiation.cc:407] Connection header received
0809 02:02:54.649782 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:54.649785 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:54.649823 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:54.649894 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:54.650494 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:54.651067 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:54.651890 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:54.652127 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:54.652241 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:54.655194 (+ 2953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:02:54.655198 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:02:54.655200 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:02:54.655225 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:02:54.656930 (+ 1705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:54.657754 (+ 824us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:54.657758 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:54.657759 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:54.657849 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:54.658207 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:54.658210 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:54.658212 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:54.658398 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:02:54.658540 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:54.658760 (+ 220us) server_negotiation.cc:299] Negotiation successful
0809 02:02:54.658862 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
W20260809 02:02:54.662391 8465 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:02:54.661306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42070 (local address 127.1.192.193:34417)
0809 02:02:54.661454 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:54.661457 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:54.661612 (+ 155us) server_negotiation.cc:407] Connection header received
0809 02:02:54.661721 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:54.661723 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:54.661760 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:54.661826 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:54.662264 (+ 438us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42070: BlockingRecv error: recv got EOF from 127.0.0.1:42070 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
W20260809 02:02:54.662796 1795 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786240941968412-1795-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786240941968412-1795-0/krb5kdc/krb5cc)
Aug 09 02:02:54 dist-test-slave-ss4n krb5kdc[7902](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240974, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:02:54.664067 1795 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 02:02:54 dist-test-slave-ss4n krb5kdc[7902](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240974, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:02:54.665238 1795 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 02:02:54 dist-test-slave-ss4n krb5kdc[7902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240974, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.192.193@KRBTEST.COM
I20260809 02:02:54.676124 8465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:54.666401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42080 (local address 127.1.192.193:34417)
0809 02:02:54.666534 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:54.666537 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:54.666754 (+ 217us) server_negotiation.cc:407] Connection header received
0809 02:02:54.666862 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:54.666864 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:54.666904 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:54.666970 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:54.667608 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:54.668126 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:54.669058 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:54.669276 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:54.669360 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:54.672123 (+ 2763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:02:54.672126 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:02:54.672128 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:02:54.672152 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:02:54.674088 (+ 1936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:54.674690 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:54.674694 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:54.674696 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:54.674830 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:54.675169 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:54.675172 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:54.675173 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:54.675359 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:02:54.675493 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:54.675863 (+ 370us) server_negotiation.cc:299] Negotiation successful
0809 02:02:54.675988 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 02:02:54.677392 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 8023
I20260809 02:02:54.685348 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 8170
I20260809 02:02:54.693977 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 8315
I20260809 02:02:54.702221 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 7917
2026-08-09T02:02:54Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2438 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:02:54 dist-test-slave-ss4n krb5kdc[8478](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:02:54 dist-test-slave-ss4n krb5kdc[8478](info): set up 2 sockets
Aug 09 02:02:54 dist-test-slave-ss4n krb5kdc[8478](info): commencing operation
krb5kdc: starting...
W20260809 02:02:55.795833 1795 mini_kdc.cc:121] Time spent starting KDC: real 1.069s user 0.007s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/krb5kdc/test-user.keytab.
Aug 09 02:02:55 dist-test-slave-ss4n krb5kdc[8478](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240975, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:02:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:02:55Z Disabled control of system clock
WARNING: no policy specified for oryx/127.1.192.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.192.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.192.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/krb5kdc/oryx_127.1.192.254.keytab.
Entry for principal oryx/127.1.192.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/krb5kdc/oryx_127.1.192.254.keytab.
I20260809 02:02:55.937286 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.192.254:34543
--webserver_interface=127.1.192.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/krb5kdc/oryx_127.1.192.254.keytab
--principal=oryx/127.1.192.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:44161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.192.254:34543
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:02:56.052778 8491 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:02:56.053051 8491 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:02:56.053092 8491 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:02:56.057247 8491 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:02:56.057324 8491 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:02:56.057344 8491 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:02:56.057361 8491 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:02:56.057376 8491 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:02:56.063460 8491 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:44161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.192.254:34543
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/krb5kdc/oryx_127.1.192.254.keytab
--principal=oryx/127.1.192.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.192.254:34543
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.192.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8491
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:02:56.064590 8491 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:02:56.065655 8491 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:02:56.074222 8496 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:02:56.074239 8499 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:02:56.074450 8491 server_base.cc:1034] running on GCE node
W20260809 02:02:56.074220 8497 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:02:56.075117 8491 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:02:56.076263 8491 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:02:56.077435 8491 hybrid_clock.cc:648] HybridClock initialized: now 1786240976077423 us; error 31 us; skew 500 ppm
Aug 09 02:02:56 dist-test-slave-ss4n krb5kdc[8478](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240976, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.192.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:02:56.080433 8491 init.cc:377] Logged in from keytab as oryx/127.1.192.254@KRBTEST.COM (short username oryx)
I20260809 02:02:56.081300 8491 webserver.cc:460] Webserver started at http://127.1.192.254:45847/ using document root <none> and password file <none>
I20260809 02:02:56.081892 8491 fs_manager.cc:359] Metadata directory not provided
I20260809 02:02:56.081959 8491 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:02:56.082130 8491 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:02:56.084190 8491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/master-0/data/instance:
uuid: "14381b53c95b4bda8c4273431f4a1762"
format_stamp: "Formatted at 2026-08-09 02:02:56 on dist-test-slave-ss4n"
server_key: "69a415e972660b629e88af69955c5962"
server_key_iv: "2383c37e572d468de49ae61e78932863"
server_key_version: "encryptionkey@0"
I20260809 02:02:56.084731 8491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/master-0/wal/instance:
uuid: "14381b53c95b4bda8c4273431f4a1762"
format_stamp: "Formatted at 2026-08-09 02:02:56 on dist-test-slave-ss4n"
server_key: "69a415e972660b629e88af69955c5962"
server_key_iv: "2383c37e572d468de49ae61e78932863"
server_key_version: "encryptionkey@0"
I20260809 02:02:56.088527 8491 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260809 02:02:56.091192 8507 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:56.092489 8491 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 02:02:56.092669 8491 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/master-0/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/master-0/wal
uuid: "14381b53c95b4bda8c4273431f4a1762"
format_stamp: "Formatted at 2026-08-09 02:02:56 on dist-test-slave-ss4n"
server_key: "69a415e972660b629e88af69955c5962"
server_key_iv: "2383c37e572d468de49ae61e78932863"
server_key_version: "encryptionkey@0"
I20260809 02:02:56.092789 8491 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:02:56.115963 8491 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:02:56.119403 8491 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:02:56.119570 8491 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:02:56.126816 8491 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.254:34543
I20260809 02:02:56.127101 8568 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.254:34543 every 8 connection(s)
I20260809 02:02:56.127861 8491 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
I20260809 02:02:56.130633 8569 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:02:56.135032 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 8491
I20260809 02:02:56.135169 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/master-0/wal/instance
I20260809 02:02:56.135474 1795 external_mini_cluster.cc:1428] Setting key 438e3fc3584c2148b4a28543bf767348
I20260809 02:02:56.136621 8569 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14381b53c95b4bda8c4273431f4a1762: Bootstrap starting.
I20260809 02:02:56.139137 8569 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 14381b53c95b4bda8c4273431f4a1762: Neither blocks nor log segments found. Creating new log.
I20260809 02:02:56.139904 8569 log.cc:824] T 00000000000000000000000000000000 P 14381b53c95b4bda8c4273431f4a1762: Log is configured to *not* fsync() on all Append() calls
I20260809 02:02:56.141911 8569 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14381b53c95b4bda8c4273431f4a1762: No bootstrap required, opened a new log
Aug 09 02:02:56 dist-test-slave-ss4n krb5kdc[8478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240975, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM
I20260809 02:02:56.145570 8569 raft_consensus.cc:348] T 00000000000000000000000000000000 P 14381b53c95b4bda8c4273431f4a1762 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14381b53c95b4bda8c4273431f4a1762" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 34543 } }
I20260809 02:02:56.145826 8569 raft_consensus.cc:374] T 00000000000000000000000000000000 P 14381b53c95b4bda8c4273431f4a1762 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:02:56.145906 8569 raft_consensus.cc:729] T 00000000000000000000000000000000 P 14381b53c95b4bda8c4273431f4a1762 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14381b53c95b4bda8c4273431f4a1762, State: Initialized, Role: FOLLOWER
I20260809 02:02:56.146406 8569 consensus_queue.cc:260] T 00000000000000000000000000000000 P 14381b53c95b4bda8c4273431f4a1762 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14381b53c95b4bda8c4273431f4a1762" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 34543 } }
I20260809 02:02:56.146564 8569 raft_consensus.cc:388] T 00000000000000000000000000000000 P 14381b53c95b4bda8c4273431f4a1762 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:02:56.146654 8569 raft_consensus.cc:482] T 00000000000000000000000000000000 P 14381b53c95b4bda8c4273431f4a1762 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:02:56.146754 8569 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 14381b53c95b4bda8c4273431f4a1762 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:02:56.147686 8569 raft_consensus.cc:504] T 00000000000000000000000000000000 P 14381b53c95b4bda8c4273431f4a1762 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14381b53c95b4bda8c4273431f4a1762" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 34543 } }
I20260809 02:02:56.148028 8569 leader_election.cc:302] T 00000000000000000000000000000000 P 14381b53c95b4bda8c4273431f4a1762 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 14381b53c95b4bda8c4273431f4a1762; no voters:
I20260809 02:02:56.148347 8569 leader_election.cc:288] T 00000000000000000000000000000000 P 14381b53c95b4bda8c4273431f4a1762 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:02:56.148483 8574 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 14381b53c95b4bda8c4273431f4a1762 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:02:56.148738 8574 raft_consensus.cc:686] T 00000000000000000000000000000000 P 14381b53c95b4bda8c4273431f4a1762 [term 1 LEADER]: Becoming Leader. State: Replica: 14381b53c95b4bda8c4273431f4a1762, State: Running, Role: LEADER
I20260809 02:02:56.149137 8574 consensus_queue.cc:237] T 00000000000000000000000000000000 P 14381b53c95b4bda8c4273431f4a1762 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14381b53c95b4bda8c4273431f4a1762" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 34543 } }
I20260809 02:02:56.149765 8569 sys_catalog.cc:564] T 00000000000000000000000000000000 P 14381b53c95b4bda8c4273431f4a1762 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:02:56.150463 8576 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14381b53c95b4bda8c4273431f4a1762 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 14381b53c95b4bda8c4273431f4a1762. Latest consensus state: current_term: 1 leader_uuid: "14381b53c95b4bda8c4273431f4a1762" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14381b53c95b4bda8c4273431f4a1762" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 34543 } } }
I20260809 02:02:56.150609 8576 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14381b53c95b4bda8c4273431f4a1762 [sys.catalog]: This master's current role is: LEADER
I20260809 02:02:56.150872 8572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:56.136967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60426 (local address 127.1.192.254:34543)
0809 02:02:56.137438 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:56.137454 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:56.137532 (+ 78us) server_negotiation.cc:407] Connection header received
0809 02:02:56.138211 (+ 679us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:56.138228 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:56.138546 (+ 318us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:56.138883 (+ 337us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:56.139705 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:56.140737 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:56.141662 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:56.142208 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:56.142385 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:56.144954 (+ 2569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:02:56.144959 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:02:56.144973 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:02:56.145001 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:02:56.147065 (+ 2064us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:56.147545 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:56.147550 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:56.147554 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:56.147667 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:56.147976 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:56.147984 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:56.147986 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:56.148522 (+ 536us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:02:56.149371 (+ 849us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:56.149543 (+ 172us) server_negotiation.cc:299] Negotiation successful
0809 02:02:56.149846 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":333,"thread_start_us":170,"threads_started":1}
I20260809 02:02:56.151309 8575 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14381b53c95b4bda8c4273431f4a1762 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "14381b53c95b4bda8c4273431f4a1762" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14381b53c95b4bda8c4273431f4a1762" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 34543 } } }
I20260809 02:02:56.151409 8575 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14381b53c95b4bda8c4273431f4a1762 [sys.catalog]: This master's current role is: LEADER
W20260809 02:02:56.155068 8589 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 14381b53c95b4bda8c4273431f4a1762: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 02:02:56.155161 8589 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 02:02:56.155224 8580 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:02:56.156085 8580 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:02:56.161918 8580 catalog_manager.cc:1347] Generated new cluster ID: 279e275c39804b08b8c01de18b06bcb9
I20260809 02:02:56.162038 8580 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:02:56.170230 8580 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:02:56.171257 8580 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:02:56.184074 8580 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 14381b53c95b4bda8c4273431f4a1762: Generated new TSK 0
I20260809 02:02:56.185021 8580 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.1.192.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.192.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.192.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/krb5kdc/oryx_127.1.192.193.keytab.
Entry for principal oryx/127.1.192.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/krb5kdc/oryx_127.1.192.193.keytab.
I20260809 02:02:56.221740 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.192.193:0
--local_ip_for_outbound_sockets=127.1.192.193
--webserver_interface=127.1.192.193
--webserver_port=0
--tserver_master_addrs=127.1.192.254:34543
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/krb5kdc/oryx_127.1.192.193.keytab
--principal=oryx/127.1.192.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:44161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:02:56.335508 8595 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:02:56.335799 8595 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:02:56.335844 8595 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:02:56.339865 8595 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:02:56.339926 8595 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:02:56.340024 8595 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.192.193
I20260809 02:02:56.345499 8595 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:44161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/krb5kdc/oryx_127.1.192.193.keytab
--principal=oryx/127.1.192.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.192.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.192.193
--webserver_port=0
--tserver_master_addrs=127.1.192.254:34543
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8595
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.192.193
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:02:56.346668 8595 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:02:56.347705 8595 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:02:56.357745 8602 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:02:56.357753 8601 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:02:56.357772 8604 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:02:56.357932 8595 server_base.cc:1034] running on GCE node
I20260809 02:02:56.358525 8595 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:02:56.359231 8595 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:02:56.360394 8595 hybrid_clock.cc:648] HybridClock initialized: now 1786240976360334 us; error 88 us; skew 500 ppm
Aug 09 02:02:56 dist-test-slave-ss4n krb5kdc[8478](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240976, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.192.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:02:56.363351 8595 init.cc:377] Logged in from keytab as oryx/127.1.192.193@KRBTEST.COM (short username oryx)
I20260809 02:02:56.364257 8595 webserver.cc:460] Webserver started at http://127.1.192.193:35199/ using document root <none> and password file <none>
I20260809 02:02:56.364840 8595 fs_manager.cc:359] Metadata directory not provided
I20260809 02:02:56.364935 8595 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:02:56.365183 8595 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:02:56.367326 8595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/ts-0/data/instance:
uuid: "2eb7b1762aec408499a7ed56b7923dfc"
format_stamp: "Formatted at 2026-08-09 02:02:56 on dist-test-slave-ss4n"
server_key: "6b107510f0ea141f7228164128d47383"
server_key_iv: "009f86906502d700fcdceed9851903a0"
server_key_version: "encryptionkey@0"
I20260809 02:02:56.367918 8595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/ts-0/wal/instance:
uuid: "2eb7b1762aec408499a7ed56b7923dfc"
format_stamp: "Formatted at 2026-08-09 02:02:56 on dist-test-slave-ss4n"
server_key: "6b107510f0ea141f7228164128d47383"
server_key_iv: "009f86906502d700fcdceed9851903a0"
server_key_version: "encryptionkey@0"
I20260809 02:02:56.371980 8595 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260809 02:02:56.374584 8612 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:56.375876 8595 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 02:02:56.376134 8595 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/ts-0/wal
uuid: "2eb7b1762aec408499a7ed56b7923dfc"
format_stamp: "Formatted at 2026-08-09 02:02:56 on dist-test-slave-ss4n"
server_key: "6b107510f0ea141f7228164128d47383"
server_key_iv: "009f86906502d700fcdceed9851903a0"
server_key_version: "encryptionkey@0"
I20260809 02:02:56.376264 8595 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:02:56.400768 8595 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:02:56.404011 8595 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:02:56.404173 8595 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:02:56.404739 8595 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:02:56.405644 8595 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:02:56.405694 8595 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:56.405735 8595 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:02:56.405751 8595 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:56.415306 8595 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.193:37543
I20260809 02:02:56.415326 8737 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.193:37543 every 8 connection(s)
I20260809 02:02:56.416250 8595 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/ts-0/data/info.pb
I20260809 02:02:56.419777 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 8595
I20260809 02:02:56.419926 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/ts-0/wal/instance
I20260809 02:02:56.420245 1795 external_mini_cluster.cc:1428] Setting key 413a5f3adac03e3558023c6b02fe59a9
Aug 09 02:02:56 dist-test-slave-ss4n krb5kdc[8478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240976, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.192.193@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM
WARNING: no policy specified for oryx/127.1.192.194@KRBTEST.COM; defaulting to no policy
I20260809 02:02:56.432521 8572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:56.418321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.193:43235 (local address 127.1.192.254:34543)
0809 02:02:56.418471 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:56.418475 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:56.419371 (+ 896us) server_negotiation.cc:407] Connection header received
0809 02:02:56.420962 (+ 1591us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:56.420965 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:56.421028 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:56.421129 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:56.422567 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:56.423109 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:56.424738 (+ 1629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:56.425003 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:56.425148 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:56.428092 (+ 2944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:02:56.428098 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:02:56.428101 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:02:56.428137 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:02:56.430036 (+ 1899us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:56.430635 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:56.430639 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:56.430643 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:56.430738 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:56.431252 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:56.431256 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:56.431258 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:56.431485 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:02:56.431616 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:56.432180 (+ 564us) server_negotiation.cc:299] Negotiation successful
0809 02:02:56.432328 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 02:02:56.433470 8740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:56.418713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:34543 (local address 127.1.192.193:43235)
0809 02:02:56.419214 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:56.419260 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 02:02:56.420722 (+ 1462us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:02:56.421353 (+ 631us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:02:56.421360 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:02:56.421910 (+ 550us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:02:56.422420 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:56.422434 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:56.423248 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:56.423252 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:56.424591 (+ 1339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:56.424605 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:56.425198 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:56.425203 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:56.425388 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:56.426144 (+ 756us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:02:56.426162 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:02:56.427890 (+ 1728us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:02:56.430191 (+ 2301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:56.430196 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:56.430209 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:56.430507 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:56.430872 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:56.430875 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:56.430877 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:56.431098 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:56.431627 (+ 529us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:02:56.431637 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:02:56.431908 (+ 271us) client_negotiation.cc:769] Sending connection context
0809 02:02:56.432135 (+ 227us) client_negotiation.cc:240] Negotiation successful
0809 02:02:56.432515 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":129,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.192.194@KRBTEST.COM" created.
I20260809 02:02:56.434978 8738 heartbeater.cc:347] Connected to a master server at 127.1.192.254:34543
I20260809 02:02:56.435271 8738 heartbeater.cc:464] Registering TS with master...
I20260809 02:02:56.435868 8738 heartbeater.cc:511] Master 127.1.192.254:34543 requested a full tablet report, sending...
I20260809 02:02:56.437563 8527 ts_manager.cc:194] Registered new tserver with Master: 2eb7b1762aec408499a7ed56b7923dfc (127.1.192.193:37543)
I20260809 02:02:56.438797 8527 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.192.193@KRBTEST.COM'} at 127.1.192.193:43235
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.192.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/krb5kdc/oryx_127.1.192.194.keytab.
Entry for principal oryx/127.1.192.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/krb5kdc/oryx_127.1.192.194.keytab.
I20260809 02:02:56.453816 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.192.194:0
--local_ip_for_outbound_sockets=127.1.192.194
--webserver_interface=127.1.192.194
--webserver_port=0
--tserver_master_addrs=127.1.192.254:34543
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/krb5kdc/oryx_127.1.192.194.keytab
--principal=oryx/127.1.192.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:44161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:02:56.568235 8745 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:02:56.568521 8745 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:02:56.568567 8745 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:02:56.572669 8745 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:02:56.572741 8745 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:02:56.572803 8745 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.192.194
I20260809 02:02:56.578487 8745 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:44161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/krb5kdc/oryx_127.1.192.194.keytab
--principal=oryx/127.1.192.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.192.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.192.194
--webserver_port=0
--tserver_master_addrs=127.1.192.254:34543
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8745
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.192.194
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:02:56.579656 8745 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:02:56.580768 8745 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:02:56.590582 8751 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:02:56.590571 8752 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:02:56.590751 8745 server_base.cc:1034] running on GCE node
W20260809 02:02:56.590575 8754 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:02:56.591349 8745 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:02:56.592041 8745 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:02:56.593262 8745 hybrid_clock.cc:648] HybridClock initialized: now 1786240976593251 us; error 39 us; skew 500 ppm
Aug 09 02:02:56 dist-test-slave-ss4n krb5kdc[8478](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240976, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.192.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:02:56.596721 8745 init.cc:377] Logged in from keytab as oryx/127.1.192.194@KRBTEST.COM (short username oryx)
I20260809 02:02:56.597712 8745 webserver.cc:460] Webserver started at http://127.1.192.194:42957/ using document root <none> and password file <none>
I20260809 02:02:56.598286 8745 fs_manager.cc:359] Metadata directory not provided
I20260809 02:02:56.598374 8745 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:02:56.598615 8745 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:02:56.600668 8745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/ts-1/data/instance:
uuid: "8030853bc1144362a198609d5c2a1a6d"
format_stamp: "Formatted at 2026-08-09 02:02:56 on dist-test-slave-ss4n"
server_key: "632d84def7d21001f400bf731ee909e1"
server_key_iv: "188ef141ac4dd8c74af4dbcb7f330f48"
server_key_version: "encryptionkey@0"
I20260809 02:02:56.601253 8745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/ts-1/wal/instance:
uuid: "8030853bc1144362a198609d5c2a1a6d"
format_stamp: "Formatted at 2026-08-09 02:02:56 on dist-test-slave-ss4n"
server_key: "632d84def7d21001f400bf731ee909e1"
server_key_iv: "188ef141ac4dd8c74af4dbcb7f330f48"
server_key_version: "encryptionkey@0"
I20260809 02:02:56.605206 8745 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260809 02:02:56.608062 8761 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:56.609598 8745 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 02:02:56.609742 8745 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/ts-1/wal
uuid: "8030853bc1144362a198609d5c2a1a6d"
format_stamp: "Formatted at 2026-08-09 02:02:56 on dist-test-slave-ss4n"
server_key: "632d84def7d21001f400bf731ee909e1"
server_key_iv: "188ef141ac4dd8c74af4dbcb7f330f48"
server_key_version: "encryptionkey@0"
I20260809 02:02:56.609875 8745 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:02:56.650413 8745 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:02:56.654034 8745 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:02:56.654253 8745 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:02:56.654887 8745 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:02:56.655776 8745 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:02:56.655845 8745 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:56.655915 8745 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:02:56.655962 8745 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:56.665295 8745 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.194:35937
I20260809 02:02:56.665318 8882 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.194:35937 every 8 connection(s)
I20260809 02:02:56.666343 8745 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/ts-1/data/info.pb
I20260809 02:02:56.672386 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 8745
I20260809 02:02:56.672477 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/ts-1/wal/instance
I20260809 02:02:56.672786 1795 external_mini_cluster.cc:1428] Setting key 4907aef4ddf83a2bde2a955934c323cb
Aug 09 02:02:56 dist-test-slave-ss4n krb5kdc[8478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240976, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.192.194@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM
I20260809 02:02:56.682120 8572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:56.668296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.194:44703 (local address 127.1.192.254:34543)
0809 02:02:56.668443 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:56.668447 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:56.669363 (+ 916us) server_negotiation.cc:407] Connection header received
0809 02:02:56.670828 (+ 1465us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:56.670831 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:56.670890 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:56.670992 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:56.672213 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:56.672809 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:56.674106 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:56.674339 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:56.674453 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:56.677247 (+ 2794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:02:56.677254 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:02:56.677258 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:02:56.677299 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:02:56.679342 (+ 2043us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:56.680134 (+ 792us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:56.680138 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:56.680139 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:56.680227 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:56.680690 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:56.680692 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:56.680694 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:56.680978 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:02:56.681062 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:56.681868 (+ 806us) server_negotiation.cc:299] Negotiation successful
0809 02:02:56.681986 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 02:02:56.683527 8886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:56.668736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:34543 (local address 127.1.192.194:44703)
0809 02:02:56.669196 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:56.669242 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 02:02:56.670602 (+ 1360us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:02:56.671190 (+ 588us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:02:56.671196 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:02:56.671566 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:02:56.672050 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:56.672064 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:56.672959 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:56.672962 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:56.673974 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:56.673984 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:56.674499 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:56.674503 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:56.674680 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:56.675339 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:02:56.675355 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:02:56.677033 (+ 1678us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:02:56.679510 (+ 2477us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:56.679520 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:56.679546 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:56.679999 (+ 453us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:56.680341 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:56.680344 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:56.680348 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:56.680586 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:56.681092 (+ 506us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:02:56.681102 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:02:56.681555 (+ 453us) client_negotiation.cc:769] Sending connection context
0809 02:02:56.681808 (+ 253us) client_negotiation.cc:240] Negotiation successful
0809 02:02:56.682298 (+ 490us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":114,"threads_started":1}
WARNING: no policy specified for oryx/127.1.192.195@KRBTEST.COM; defaulting to no policy
I20260809 02:02:56.684904 8883 heartbeater.cc:347] Connected to a master server at 127.1.192.254:34543
I20260809 02:02:56.685195 8883 heartbeater.cc:464] Registering TS with master...
I20260809 02:02:56.685894 8883 heartbeater.cc:511] Master 127.1.192.254:34543 requested a full tablet report, sending...
I20260809 02:02:56.687150 8527 ts_manager.cc:194] Registered new tserver with Master: 8030853bc1144362a198609d5c2a1a6d (127.1.192.194:35937)
I20260809 02:02:56.687870 8527 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.192.194@KRBTEST.COM'} at 127.1.192.194:44703
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.192.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.192.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/krb5kdc/oryx_127.1.192.195.keytab.
Entry for principal oryx/127.1.192.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/krb5kdc/oryx_127.1.192.195.keytab.
I20260809 02:02:56.706794 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.192.195:0
--local_ip_for_outbound_sockets=127.1.192.195
--webserver_interface=127.1.192.195
--webserver_port=0
--tserver_master_addrs=127.1.192.254:34543
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/krb5kdc/oryx_127.1.192.195.keytab
--principal=oryx/127.1.192.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:44161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:02:56.821786 8889 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:02:56.822098 8889 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:02:56.822197 8889 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:02:56.826390 8889 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:02:56.826499 8889 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:02:56.826606 8889 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.192.195
I20260809 02:02:56.832266 8889 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:44161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/krb5kdc/oryx_127.1.192.195.keytab
--principal=oryx/127.1.192.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.192.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.192.195
--webserver_port=0
--tserver_master_addrs=127.1.192.254:34543
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8889
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.192.195
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:02:56.833537 8889 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:02:56.834627 8889 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:02:56.845707 8897 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:02:56.845707 8894 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:02:56.845747 8895 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:02:56.846061 8889 server_base.cc:1034] running on GCE node
I20260809 02:02:56.846650 8889 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:02:56.847383 8889 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:02:56.848760 8889 hybrid_clock.cc:648] HybridClock initialized: now 1786240976848739 us; error 44 us; skew 500 ppm
Aug 09 02:02:56 dist-test-slave-ss4n krb5kdc[8478](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240976, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.192.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:02:56.852528 8889 init.cc:377] Logged in from keytab as oryx/127.1.192.195@KRBTEST.COM (short username oryx)
I20260809 02:02:56.853801 8889 webserver.cc:460] Webserver started at http://127.1.192.195:45817/ using document root <none> and password file <none>
I20260809 02:02:56.854470 8889 fs_manager.cc:359] Metadata directory not provided
I20260809 02:02:56.854578 8889 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:02:56.854842 8889 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:02:56.857126 8889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/ts-2/data/instance:
uuid: "cd2035d243e94eeea383bb2f79f51dc1"
format_stamp: "Formatted at 2026-08-09 02:02:56 on dist-test-slave-ss4n"
server_key: "96da89dd767bab8c7ca9c8146f1b2cc5"
server_key_iv: "3f11469d24ed1d6f316e96710f3a7854"
server_key_version: "encryptionkey@0"
I20260809 02:02:56.857787 8889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/ts-2/wal/instance:
uuid: "cd2035d243e94eeea383bb2f79f51dc1"
format_stamp: "Formatted at 2026-08-09 02:02:56 on dist-test-slave-ss4n"
server_key: "96da89dd767bab8c7ca9c8146f1b2cc5"
server_key_iv: "3f11469d24ed1d6f316e96710f3a7854"
server_key_version: "encryptionkey@0"
I20260809 02:02:56.861990 8889 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260809 02:02:56.864692 8905 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:56.866153 8889 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 02:02:56.866304 8889 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/ts-2/wal
uuid: "cd2035d243e94eeea383bb2f79f51dc1"
format_stamp: "Formatted at 2026-08-09 02:02:56 on dist-test-slave-ss4n"
server_key: "96da89dd767bab8c7ca9c8146f1b2cc5"
server_key_iv: "3f11469d24ed1d6f316e96710f3a7854"
server_key_version: "encryptionkey@0"
I20260809 02:02:56.866433 8889 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:02:56.887285 8889 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:02:56.890692 8889 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:02:56.890924 8889 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:02:56.891743 8889 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:02:56.892845 8889 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:02:56.892925 8889 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:56.892998 8889 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:02:56.893031 8889 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:56.903304 8889 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.195:39329
I20260809 02:02:56.903326 9035 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.195:39329 every 8 connection(s)
I20260809 02:02:56.904320 8889 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/ts-2/data/info.pb
I20260809 02:02:56.904654 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 8889
I20260809 02:02:56.904776 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/ts-2/wal/instance
I20260809 02:02:56.905130 1795 external_mini_cluster.cc:1428] Setting key bcf0a3f75c5181a65683e23e453106ef
Aug 09 02:02:56 dist-test-slave-ss4n krb5kdc[8478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240976, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.192.195@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM
I20260809 02:02:56.921417 8572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:56.906444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.195:57737 (local address 127.1.192.254:34543)
0809 02:02:56.906618 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:56.906623 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:56.907883 (+ 1260us) server_negotiation.cc:407] Connection header received
0809 02:02:56.909370 (+ 1487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:56.909373 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:56.909438 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:56.909594 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:56.910884 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:56.911494 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:56.912914 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:56.913266 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:56.913404 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:56.916665 (+ 3261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:02:56.916669 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:02:56.916672 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:02:56.916708 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:02:56.918409 (+ 1701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:56.919250 (+ 841us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:56.919256 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:56.919261 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:56.919366 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:56.919818 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:56.919822 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:56.919825 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:56.920134 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:02:56.920216 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:56.921060 (+ 844us) server_negotiation.cc:299] Negotiation successful
0809 02:02:56.921243 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 02:02:56.922487 9038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:56.907039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:34543 (local address 127.1.192.195:57737)
0809 02:02:56.907714 (+ 675us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:56.907780 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0809 02:02:56.909137 (+ 1357us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:02:56.909800 (+ 663us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:02:56.909807 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:02:56.910201 (+ 394us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:02:56.910722 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:56.910738 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:56.911717 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:56.911720 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:56.912722 (+ 1002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:56.912733 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:56.913455 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:56.913459 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:56.913845 (+ 386us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:56.914749 (+ 904us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:02:56.914768 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:02:56.916459 (+ 1691us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:02:56.918725 (+ 2266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:56.918732 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:56.918751 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:56.919132 (+ 381us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:56.919479 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:56.919482 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:56.919484 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:56.919670 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:56.920309 (+ 639us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:02:56.920316 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:02:56.920679 (+ 363us) client_negotiation.cc:769] Sending connection context
0809 02:02:56.920960 (+ 281us) client_negotiation.cc:240] Negotiation successful
0809 02:02:56.921349 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":141,"threads_started":1}
I20260809 02:02:56.923983 9036 heartbeater.cc:347] Connected to a master server at 127.1.192.254:34543
I20260809 02:02:56.924242 9036 heartbeater.cc:464] Registering TS with master...
I20260809 02:02:56.924950 9036 heartbeater.cc:511] Master 127.1.192.254:34543 requested a full tablet report, sending...
I20260809 02:02:56.926403 8527 ts_manager.cc:194] Registered new tserver with Master: cd2035d243e94eeea383bb2f79f51dc1 (127.1.192.195:39329)
I20260809 02:02:56.927202 8527 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.192.195@KRBTEST.COM'} at 127.1.192.195:57737
I20260809 02:02:56.931731 1795 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.1.192.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.192.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.192.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/krb5kdc/oryx_127.1.192.253.keytab.
Entry for principal oryx/127.1.192.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/krb5kdc/oryx_127.1.192.253.keytab.
I20260809 02:02:56.962002 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.192.253:40841
--webserver_interface=127.1.192.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/krb5kdc/oryx_127.1.192.253.keytab
--principal=oryx/127.1.192.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:44161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.192.254:34543,127.1.192.253:40841
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:02:57.075896 9041 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:02:57.076213 9041 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:02:57.076296 9041 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:02:57.080396 9041 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:02:57.080477 9041 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:02:57.080514 9041 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:02:57.080572 9041 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:02:57.080624 9041 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:02:57.086117 9041 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:44161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.1.192.254:34543,127.1.192.253:40841
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/krb5kdc/oryx_127.1.192.253.keytab
--principal=oryx/127.1.192.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.192.253:40841
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.1.192.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9041
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
W20260809 02:02:57.086387 9041 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.1.192.254:34543,127.1.192.253:40841'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260809 02:02:57.087322 9041 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:02:57.088357 9041 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:02:57.096971 9047 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:02:57.097070 9048 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:02:57.097101 9050 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:02:57.097383 9041 server_base.cc:1034] running on GCE node
I20260809 02:02:57.098034 9041 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:02:57.099344 9041 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:02:57.100594 9041 hybrid_clock.cc:648] HybridClock initialized: now 1786240977100573 us; error 42 us; skew 500 ppm
Aug 09 02:02:57 dist-test-slave-ss4n krb5kdc[8478](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240977, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.192.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:02:57.103727 9041 init.cc:377] Logged in from keytab as oryx/127.1.192.253@KRBTEST.COM (short username oryx)
I20260809 02:02:57.104758 9041 webserver.cc:460] Webserver started at http://127.1.192.253:44069/ using document root <none> and password file <none>
I20260809 02:02:57.105348 9041 fs_manager.cc:359] Metadata directory not provided
I20260809 02:02:57.105450 9041 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:02:57.105782 9041 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:02:57.107995 9041 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/master-1/data/instance:
uuid: "bf6ddc1ba97745d897049f1300717b5b"
format_stamp: "Formatted at 2026-08-09 02:02:57 on dist-test-slave-ss4n"
server_key: "c41a2f46376462a3e0d4f6e4012c9e39"
server_key_iv: "bf7321cf42f017e106b35b859266f388"
server_key_version: "encryptionkey@0"
I20260809 02:02:57.108623 9041 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/master-1/wal/instance:
uuid: "bf6ddc1ba97745d897049f1300717b5b"
format_stamp: "Formatted at 2026-08-09 02:02:57 on dist-test-slave-ss4n"
server_key: "c41a2f46376462a3e0d4f6e4012c9e39"
server_key_iv: "bf7321cf42f017e106b35b859266f388"
server_key_version: "encryptionkey@0"
I20260809 02:02:57.112805 9041 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260809 02:02:57.115585 9058 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:57.116968 9041 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 02:02:57.117136 9041 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/master-1/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/master-1/wal
uuid: "bf6ddc1ba97745d897049f1300717b5b"
format_stamp: "Formatted at 2026-08-09 02:02:57 on dist-test-slave-ss4n"
server_key: "c41a2f46376462a3e0d4f6e4012c9e39"
server_key_iv: "bf7321cf42f017e106b35b859266f388"
server_key_version: "encryptionkey@0"
I20260809 02:02:57.117266 9041 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:02:57.134825 9041 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:02:57.138412 9041 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:02:57.138646 9041 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:02:57.146713 9041 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.253:40841
I20260809 02:02:57.146907 9117 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.253:40841 every 8 connection(s)
I20260809 02:02:57.147845 9041 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/master-1/data/info.pb
I20260809 02:02:57.149740 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 9041
I20260809 02:02:57.151055 9118 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:02:57.154911 9118 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.192.254" port: 34543 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 02:02:57 dist-test-slave-ss4n krb5kdc[8478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240975, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.192.253@KRBTEST.COM
Aug 09 02:02:57 dist-test-slave-ss4n krb5kdc[8478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240977, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.192.253@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM
I20260809 02:02:57.165658 9121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:57.151462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48526 (local address 127.1.192.253:40841)
0809 02:02:57.151903 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:57.151927 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:57.151964 (+ 37us) server_negotiation.cc:407] Connection header received
0809 02:02:57.152679 (+ 715us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:57.152697 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:57.153007 (+ 310us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:57.153340 (+ 333us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:57.154227 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:57.155365 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:57.156229 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:57.156906 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:57.157060 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:57.159772 (+ 2712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:02:57.159778 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:02:57.159786 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:02:57.159814 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:02:57.162050 (+ 2236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:57.162553 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:57.162559 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:57.162565 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:57.162710 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:57.163136 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:57.163140 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:57.163142 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:57.163776 (+ 634us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:02:57.163999 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:57.164272 (+ 273us) server_negotiation.cc:299] Negotiation successful
0809 02:02:57.164555 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":174,"threads_started":1}
I20260809 02:02:57.166519 8572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:57.156604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60438 (local address 127.1.192.254:34543)
0809 02:02:57.156766 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:57.156773 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:57.157176 (+ 403us) server_negotiation.cc:407] Connection header received
0809 02:02:57.157373 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:57.157376 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:57.157426 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:57.157538 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:57.158182 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:57.158702 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:57.159687 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:57.159919 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:57.160031 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:57.162450 (+ 2419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:02:57.162455 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:02:57.162463 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:02:57.162488 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:02:57.164246 (+ 1758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:57.164787 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:57.164791 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:57.164793 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:57.164887 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:57.165306 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:57.165309 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:57.165311 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:57.165641 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:02:57.165756 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:57.166243 (+ 487us) server_negotiation.cc:299] Negotiation successful
0809 02:02:57.166371 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 02:02:57.166597 9122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:57.156589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:34543 (local address 127.0.0.1:60438)
0809 02:02:57.157049 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:57.157097 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 02:02:57.157264 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:02:57.157582 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:02:57.157587 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:02:57.157934 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:02:57.158052 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:57.158058 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:57.158845 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:57.158848 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:57.159562 (+ 714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:57.159572 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:57.160035 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:57.160038 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:57.160168 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:57.160590 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:02:57.160601 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:02:57.162297 (+ 1696us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:02:57.164373 (+ 2076us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:57.164384 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:57.164392 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:57.164680 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:57.165029 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:57.165032 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:57.165033 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:57.165152 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:57.165833 (+ 681us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:02:57.165844 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:02:57.166003 (+ 159us) client_negotiation.cc:769] Sending connection context
0809 02:02:57.166162 (+ 159us) client_negotiation.cc:240] Negotiation successful
0809 02:02:57.166340 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":173,"threads_started":1}
I20260809 02:02:57.168180 9118 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.192.253" port: 40841 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 02:02:57 dist-test-slave-ss4n krb5kdc[8478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240977, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.192.253@KRBTEST.COM for oryx/127.1.192.253@KRBTEST.COM
I20260809 02:02:57.177304 9122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:57.168695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.253:40841 (local address 127.0.0.1:48538)
0809 02:02:57.168898 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:57.168939 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 02:02:57.169050 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:02:57.169276 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:02:57.169278 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:02:57.169595 (+ 317us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:02:57.169720 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:57.169727 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:57.171069 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:57.171072 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:57.171716 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:57.171725 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:57.172139 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:57.172142 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:57.172214 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:57.172792 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:02:57.172804 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:02:57.174237 (+ 1433us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:02:57.176038 (+ 1801us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:57.176040 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:57.176042 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:57.176235 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:57.176466 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:57.176469 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:57.176470 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:57.176539 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:57.176893 (+ 354us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:02:57.176895 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:02:57.176973 (+ 78us) client_negotiation.cc:769] Sending connection context
0809 02:02:57.177061 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 02:02:57.177177 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67}
I20260809 02:02:57.177416 9121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:57.168815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48538 (local address 127.1.192.253:40841)
0809 02:02:57.168921 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:57.168927 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:57.168994 (+ 67us) server_negotiation.cc:407] Connection header received
0809 02:02:57.169158 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:57.169161 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:57.169206 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:57.169262 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:57.169832 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:57.170421 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:57.171820 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:57.172039 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:57.172119 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:57.174385 (+ 2266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:02:57.174388 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:02:57.174389 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:02:57.174410 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:02:57.175940 (+ 1530us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:57.176317 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:57.176320 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:57.176321 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:57.176389 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:57.176639 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:57.176641 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:57.176642 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:57.176806 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:02:57.176878 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:57.177180 (+ 302us) server_negotiation.cc:299] Negotiation successful
0809 02:02:57.177277 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 02:02:57.180429 9118 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf6ddc1ba97745d897049f1300717b5b: Bootstrap starting.
I20260809 02:02:57.182793 9118 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bf6ddc1ba97745d897049f1300717b5b: Neither blocks nor log segments found. Creating new log.
I20260809 02:02:57.183497 9118 log.cc:824] T 00000000000000000000000000000000 P bf6ddc1ba97745d897049f1300717b5b: Log is configured to *not* fsync() on all Append() calls
I20260809 02:02:57.185683 9118 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf6ddc1ba97745d897049f1300717b5b: No bootstrap required, opened a new log
I20260809 02:02:57.188742 9118 raft_consensus.cc:348] T 00000000000000000000000000000000 P bf6ddc1ba97745d897049f1300717b5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14381b53c95b4bda8c4273431f4a1762" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 34543 } } peers { permanent_uuid: "bf6ddc1ba97745d897049f1300717b5b" member_type: VOTER last_known_addr { host: "127.1.192.253" port: 40841 } }
I20260809 02:02:57.188941 9118 raft_consensus.cc:374] T 00000000000000000000000000000000 P bf6ddc1ba97745d897049f1300717b5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:02:57.189024 9118 raft_consensus.cc:729] T 00000000000000000000000000000000 P bf6ddc1ba97745d897049f1300717b5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf6ddc1ba97745d897049f1300717b5b, State: Initialized, Role: FOLLOWER
I20260809 02:02:57.189433 9118 consensus_queue.cc:260] T 00000000000000000000000000000000 P bf6ddc1ba97745d897049f1300717b5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14381b53c95b4bda8c4273431f4a1762" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 34543 } } peers { permanent_uuid: "bf6ddc1ba97745d897049f1300717b5b" member_type: VOTER last_known_addr { host: "127.1.192.253" port: 40841 } }
I20260809 02:02:57.189972 9124 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf6ddc1ba97745d897049f1300717b5b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14381b53c95b4bda8c4273431f4a1762" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 34543 } } peers { permanent_uuid: "bf6ddc1ba97745d897049f1300717b5b" member_type: VOTER last_known_addr { host: "127.1.192.253" port: 40841 } } }
I20260809 02:02:57.190083 9124 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf6ddc1ba97745d897049f1300717b5b [sys.catalog]: This master's current role is: FOLLOWER
I20260809 02:02:57.190696 9118 sys_catalog.cc:564] T 00000000000000000000000000000000 P bf6ddc1ba97745d897049f1300717b5b [sys.catalog]: configured and running, proceeding with master startup.
W20260809 02:02:57.195890 9136 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bf6ddc1ba97745d897049f1300717b5b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 02:02:57.196024 9136 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 02:02:57.198165 9041 master_runner.cc:428] Detected that this master bf6ddc1ba97745d897049f1300717b5b is joining an existing cluster
I20260809 02:02:57.198218 9041 master_runner.cc:432] Initiating AddMaster RPC to add 127.1.192.253:40841
I20260809 02:02:57.209571 9121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:57.200862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48542 (local address 127.1.192.253:40841)
0809 02:02:57.201021 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:57.201026 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:57.201103 (+ 77us) server_negotiation.cc:407] Connection header received
0809 02:02:57.201309 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:57.201313 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:57.201369 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:57.201506 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:57.202231 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:57.202939 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:57.203796 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:57.204082 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:57.204190 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:57.205347 (+ 1157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:02:57.205351 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:02:57.205355 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:02:57.205383 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:02:57.207359 (+ 1976us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:57.207858 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:57.207862 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:57.207865 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:57.207977 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:57.208348 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:57.208352 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:57.208354 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:57.208603 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:02:57.208733 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:57.209178 (+ 445us) server_negotiation.cc:299] Negotiation successful
0809 02:02:57.209338 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":38,"server-negotiator.queue_time_us":85}
I20260809 02:02:57.209764 9145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:57.200292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:34543 (local address 127.0.0.1:60444)
0809 02:02:57.200560 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:57.200577 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 02:02:57.200685 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:02:57.201476 (+ 791us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:02:57.201512 (+ 36us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:02:57.201778 (+ 266us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:02:57.201897 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:57.201903 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:57.202982 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:57.202986 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:57.204112 (+ 1126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:57.204125 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:57.204956 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:57.204961 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:57.205069 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:57.205607 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:02:57.205621 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:02:57.206128 (+ 507us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:02:57.208125 (+ 1997us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:57.208128 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:57.208129 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:57.208403 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:57.208684 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:57.208689 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:57.208691 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:57.208778 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:57.209160 (+ 382us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:02:57.209163 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:02:57.209259 (+ 96us) client_negotiation.cc:769] Sending connection context
0809 02:02:57.209366 (+ 107us) client_negotiation.cc:240] Negotiation successful
0809 02:02:57.209525 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"mutex_wait_us":53,"spinlock_wait_cycles":1280,"thread_start_us":110,"threads_started":1}
I20260809 02:02:57.210232 8572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:57.201125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60444 (local address 127.1.192.254:34543)
0809 02:02:57.201277 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:57.201281 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:57.201293 (+ 12us) server_negotiation.cc:407] Connection header received
0809 02:02:57.201336 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:57.201338 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:57.201374 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:57.201461 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:57.202017 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:57.202846 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:57.204290 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:57.204662 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:57.204760 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:57.206272 (+ 1512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:02:57.206276 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:02:57.206278 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:02:57.206306 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:02:57.208001 (+ 1695us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:57.208503 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:57.208507 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:57.208509 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:57.208593 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:57.208869 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:57.208874 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:57.208876 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:57.209056 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:02:57.209171 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:57.209420 (+ 249us) server_negotiation.cc:299] Negotiation successful
0809 02:02:57.209574 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 02:02:57.210628 9146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:57.200742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.253:40841 (local address 127.0.0.1:48542)
0809 02:02:57.201012 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:57.201023 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 02:02:57.201133 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:02:57.201656 (+ 523us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:02:57.201659 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:02:57.201949 (+ 290us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:02:57.202093 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:57.202103 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:57.203053 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:57.203056 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:57.203682 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:57.203690 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:57.204191 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:57.204194 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:57.204277 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:57.204820 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:02:57.204841 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:02:57.205191 (+ 350us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:02:57.207483 (+ 2292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:57.207489 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:57.207493 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:57.207734 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:57.208104 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:57.208108 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:57.208111 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:57.208235 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:57.208779 (+ 544us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:02:57.208784 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:02:57.208914 (+ 130us) client_negotiation.cc:769] Sending connection context
0809 02:02:57.210127 (+ 1213us) client_negotiation.cc:240] Negotiation successful
0809 02:02:57.210415 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":208,"mutex_wait_us":23,"thread_start_us":107,"threads_started":1}
Aug 09 02:02:57 dist-test-slave-ss4n krb5kdc[8478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240976, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.192.254@KRBTEST.COM for oryx/127.1.192.253@KRBTEST.COM
I20260809 02:02:57.224807 9121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:57.214844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48552 (local address 127.1.192.253:40841)
0809 02:02:57.214957 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:57.214961 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:57.215362 (+ 401us) server_negotiation.cc:407] Connection header received
0809 02:02:57.215580 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:57.215585 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:57.215657 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:57.215748 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:57.216419 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:57.216911 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:57.217873 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:57.218104 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:57.218212 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:57.220620 (+ 2408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:02:57.220624 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:02:57.220626 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:02:57.220653 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:02:57.222656 (+ 2003us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:57.223312 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:57.223315 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:57.223316 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:57.223412 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:57.223750 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:57.223753 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:57.223754 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:57.224037 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:02:57.224171 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:57.224511 (+ 340us) server_negotiation.cc:299] Negotiation successful
0809 02:02:57.224644 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 02:02:57.224822 9147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:57.214831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.253:40841 (local address 127.0.0.1:48552)
0809 02:02:57.215237 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:57.215281 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 02:02:57.215450 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:02:57.215783 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:02:57.215789 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:02:57.216167 (+ 378us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:02:57.216281 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:57.216288 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:57.217019 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:57.217023 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:57.217748 (+ 725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:57.217758 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:57.218215 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:57.218219 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:57.218336 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:57.218947 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:02:57.218959 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:02:57.220472 (+ 1513us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:02:57.222792 (+ 2320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:57.222804 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:57.222811 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:57.223185 (+ 374us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:57.223525 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:57.223529 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:57.223531 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:57.223644 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:57.224172 (+ 528us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:02:57.224180 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:02:57.224305 (+ 125us) client_negotiation.cc:769] Sending connection context
0809 02:02:57.224458 (+ 153us) client_negotiation.cc:240] Negotiation successful
0809 02:02:57.224658 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":130,"threads_started":1}
I20260809 02:02:57.226768 8527 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.1.192.253:40841: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "bf6ddc1ba97745d897049f1300717b5b"
member_type: NON_VOTER
last_known_addr {
host: "127.1.192.253"
port: 40841
}
attrs {
promote: true
}
}
dest_uuid: "14381b53c95b4bda8c4273431f4a1762"
cas_config_opid_index: -1
I20260809 02:02:57.227540 8527 consensus_queue.cc:237] T 00000000000000000000000000000000 P 14381b53c95b4bda8c4273431f4a1762 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "14381b53c95b4bda8c4273431f4a1762" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 34543 } } peers { permanent_uuid: "bf6ddc1ba97745d897049f1300717b5b" member_type: NON_VOTER last_known_addr { host: "127.1.192.253" port: 40841 } attrs { promote: true } }
I20260809 02:02:57.229656 9151 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 14381b53c95b4bda8c4273431f4a1762 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER bf6ddc1ba97745d897049f1300717b5b (127.1.192.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "14381b53c95b4bda8c4273431f4a1762" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 34543 } } peers { permanent_uuid: "bf6ddc1ba97745d897049f1300717b5b" member_type: NON_VOTER last_known_addr { host: "127.1.192.253" port: 40841 } attrs { promote: true } } }
I20260809 02:02:57.230404 9089 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bf6ddc1ba97745d897049f1300717b5b [term 0 FOLLOWER]: Advancing to term 1
WARNING: no policy specified for oryx/127.1.192.252@KRBTEST.COM; defaulting to no policy
I20260809 02:02:57.231865 9089 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bf6ddc1ba97745d897049f1300717b5b [term 1 FOLLOWER]: Refusing update from remote peer 14381b53c95b4bda8c4273431f4a1762: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260809 02:02:57.232038 9151 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.1.192.253:40841
I20260809 02:02:57.232676 9041 master.cc:537] Master@127.1.192.253:40841 shutting down...
I20260809 02:02:57.232686 9124 raft_consensus.cc:482] T 00000000000000000000000000000000 P bf6ddc1ba97745d897049f1300717b5b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:02:57.232717 9153 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14381b53c95b4bda8c4273431f4a1762 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "14381b53c95b4bda8c4273431f4a1762" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "14381b53c95b4bda8c4273431f4a1762" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 34543 } } peers { permanent_uuid: "bf6ddc1ba97745d897049f1300717b5b" member_type: NON_VOTER last_known_addr { host: "127.1.192.253" port: 40841 } attrs { promote: true } } }
I20260809 02:02:57.232856 9153 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14381b53c95b4bda8c4273431f4a1762 [sys.catalog]: This master's current role is: LEADER
I20260809 02:02:57.232838 9124 raft_consensus.cc:504] T 00000000000000000000000000000000 P bf6ddc1ba97745d897049f1300717b5b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14381b53c95b4bda8c4273431f4a1762" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 34543 } } peers { permanent_uuid: "bf6ddc1ba97745d897049f1300717b5b" member_type: VOTER last_known_addr { host: "127.1.192.253" port: 40841 } }
I20260809 02:02:57.233108 9153 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 14381b53c95b4bda8c4273431f4a1762 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf6ddc1ba97745d897049f1300717b5b" member_type: NON_VOTER last_known_addr { host: "127.1.192.253" port: 40841 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:02:57.234169 9124 leader_election.cc:288] T 00000000000000000000000000000000 P bf6ddc1ba97745d897049f1300717b5b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 14381b53c95b4bda8c4273431f4a1762 (127.1.192.254:34543)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.192.252@KRBTEST.COM" created.
I20260809 02:02:57.235329 8539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bf6ddc1ba97745d897049f1300717b5b" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14381b53c95b4bda8c4273431f4a1762" is_pre_election: true
I20260809 02:02:57.235997 9060 leader_election.cc:302] T 00000000000000000000000000000000 P bf6ddc1ba97745d897049f1300717b5b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: bf6ddc1ba97745d897049f1300717b5b; no voters: 14381b53c95b4bda8c4273431f4a1762
I20260809 02:02:57.236665 9124 raft_consensus.cc:2726] T 00000000000000000000000000000000 P bf6ddc1ba97745d897049f1300717b5b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260809 02:02:57.237928 8509 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260809 02:02:57.238878 8509 consensus_peers.cc:487] T 00000000000000000000000000000000 P 14381b53c95b4bda8c4273431f4a1762 -> Peer bf6ddc1ba97745d897049f1300717b5b (127.1.192.253:40841): Couldn't send request to peer bf6ddc1ba97745d897049f1300717b5b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260809 02:02:57.239825 9041 raft_consensus.cc:2229] T 00000000000000000000000000000000 P bf6ddc1ba97745d897049f1300717b5b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:02:57.240093 9041 raft_consensus.cc:2258] T 00000000000000000000000000000000 P bf6ddc1ba97745d897049f1300717b5b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:02:57.240197 9041 tablet_replica.cc:330] T 00000000000000000000000000000000 P bf6ddc1ba97745d897049f1300717b5b: stopping tablet replica
I20260809 02:02:57.255612 9041 master.cc:552] Master@127.1.192.253:40841 shutdown complete.
I20260809 02:02:57.255673 9041 master_runner.cc:305] Clearing existing system tablet
I20260809 02:02:57.256031 9041 fs_manager.cc:359] Metadata directory not provided
I20260809 02:02:57.256108 9041 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:02:57.258512 9041 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.192.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/krb5kdc/oryx_127.1.192.252.keytab.
Entry for principal oryx/127.1.192.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/krb5kdc/oryx_127.1.192.252.keytab.
I20260809 02:02:57.260458 9158 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:57.260726 9041 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:02:57.260885 9041 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:02:57.260923 9041 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/master-1/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/master-1/wal
uuid: "bf6ddc1ba97745d897049f1300717b5b"
format_stamp: "Formatted at 2026-08-09 02:02:57 on dist-test-slave-ss4n"
server_key: "c41a2f46376462a3e0d4f6e4012c9e39"
server_key_iv: "bf7321cf42f017e106b35b859266f388"
server_key_version: "encryptionkey@0"
I20260809 02:02:57.261157 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.192.252:44581
--webserver_interface=127.1.192.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/krb5kdc/oryx_127.1.192.252.keytab
--principal=oryx/127.1.192.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:44161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.192.254:34543,127.1.192.253:40841,127.1.192.252:44581
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 02:02:57.263044 9041 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P bf6ddc1ba97745d897049f1300717b5b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:02:57.266110 9041 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P bf6ddc1ba97745d897049f1300717b5b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260809 02:02:57.266248 9041 log.cc:1196] T 00000000000000000000000000000000 P bf6ddc1ba97745d897049f1300717b5b: Deleting WAL directory at /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260809 02:02:57.266561 9041 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P bf6ddc1ba97745d897049f1300717b5b: Deleting consensus metadata
I20260809 02:02:57.266769 9041 master_runner.cc:315] Copying system tablet from 127.1.192.254:34543
I20260809 02:02:57.268430 9041 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P bf6ddc1ba97745d897049f1300717b5b: tablet copy: Beginning tablet copy session from remote peer at address 127.1.192.254:34543
I20260809 02:02:57.279790 8572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:57.269317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60456 (local address 127.1.192.254:34543)
0809 02:02:57.269468 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:57.269474 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:57.269727 (+ 253us) server_negotiation.cc:407] Connection header received
0809 02:02:57.269886 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:57.269890 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:57.269966 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:57.270092 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:57.270839 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:57.271837 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:57.273542 (+ 1705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:57.273946 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:57.274080 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:57.275619 (+ 1539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:02:57.275624 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:02:57.275627 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:02:57.275663 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:02:57.277736 (+ 2073us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:57.278271 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:57.278276 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:57.278278 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:57.278389 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:57.278704 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:57.278708 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:57.278710 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:57.278988 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:02:57.279108 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:57.279427 (+ 319us) server_negotiation.cc:299] Negotiation successful
0809 02:02:57.279587 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 02:02:57.279805 9165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:57.269201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:34543 (local address 127.0.0.1:60456)
0809 02:02:57.269603 (+ 402us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:57.269622 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 02:02:57.269765 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:02:57.270143 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:02:57.270148 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:02:57.270430 (+ 282us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:02:57.270679 (+ 249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:57.270692 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:57.272202 (+ 1510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:57.272207 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:57.273334 (+ 1127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:57.273350 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:57.274114 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:57.274120 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:57.274252 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:57.274942 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:02:57.274962 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:02:57.275433 (+ 471us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:02:57.277864 (+ 2431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:57.277868 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:57.277870 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:57.278154 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:57.278502 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:57.278505 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:57.278507 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:57.278602 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:57.279111 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:02:57.279114 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:02:57.279221 (+ 107us) client_negotiation.cc:769] Sending connection context
0809 02:02:57.279332 (+ 111us) client_negotiation.cc:240] Negotiation successful
0809 02:02:57.279578 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":108,"threads_started":1}
I20260809 02:02:57.281069 8552 tablet_copy_service.cc:140] P 14381b53c95b4bda8c4273431f4a1762: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer bf6ddc1ba97745d897049f1300717b5b ({username='oryx', principal='oryx/127.1.192.253@KRBTEST.COM'} at 127.0.0.1:60456)
I20260809 02:02:57.281275 8552 tablet_copy_service.cc:161] P 14381b53c95b4bda8c4273431f4a1762: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer bf6ddc1ba97745d897049f1300717b5b at {username='oryx', principal='oryx/127.1.192.253@KRBTEST.COM'} at 127.0.0.1:60456: session id = bf6ddc1ba97745d897049f1300717b5b-00000000000000000000000000000000
I20260809 02:02:57.283411 8552 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 14381b53c95b4bda8c4273431f4a1762: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 02:02:57.285636 9041 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260809 02:02:57.288520 9041 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P bf6ddc1ba97745d897049f1300717b5b: tablet copy: Starting download of 0 data blocks...
I20260809 02:02:57.288761 9041 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P bf6ddc1ba97745d897049f1300717b5b: tablet copy: Starting download of 1 WAL segments...
I20260809 02:02:57.290887 9041 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P bf6ddc1ba97745d897049f1300717b5b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 02:02:57.293538 8552 tablet_copy_service.cc:342] P 14381b53c95b4bda8c4273431f4a1762: Request end of tablet copy session bf6ddc1ba97745d897049f1300717b5b-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.1.192.253@KRBTEST.COM'} at 127.0.0.1:60456
I20260809 02:02:57.293668 8552 tablet_copy_service.cc:433] P 14381b53c95b4bda8c4273431f4a1762: ending tablet copy session bf6ddc1ba97745d897049f1300717b5b-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer bf6ddc1ba97745d897049f1300717b5b
I20260809 02:02:57.303289 9041 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:02:57.305150 9166 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:02:57.305375 9169 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:02:57.305166 9167 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:02:57.306047 9041 server_base.cc:1034] running on GCE node
I20260809 02:02:57.306216 9041 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:02:57.306530 9041 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:02:57.307683 9041 hybrid_clock.cc:648] HybridClock initialized: now 1786240977307657 us; error 44 us; skew 500 ppm
Aug 09 02:02:57 dist-test-slave-ss4n krb5kdc[8478](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240977, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.192.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:02:57.309514 9041 init.cc:377] Logged in from keytab as oryx/127.1.192.253@KRBTEST.COM (short username oryx)
I20260809 02:02:57.310067 9041 webserver.cc:460] Webserver started at http://127.1.192.253:38875/ using document root <none> and password file <none>
I20260809 02:02:57.310309 9041 fs_manager.cc:359] Metadata directory not provided
I20260809 02:02:57.310356 9041 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:02:57.311823 9041 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:02:57.312943 9176 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:57.313252 9041 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:02:57.313393 9041 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/master-1/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/master-1/wal
uuid: "bf6ddc1ba97745d897049f1300717b5b"
format_stamp: "Formatted at 2026-08-09 02:02:57 on dist-test-slave-ss4n"
server_key: "c41a2f46376462a3e0d4f6e4012c9e39"
server_key_iv: "bf7321cf42f017e106b35b859266f388"
server_key_version: "encryptionkey@0"
I20260809 02:02:57.313512 9041 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:02:57.323760 9041 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:02:57.326179 9041 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:02:57.331494 9041 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.253:40841
I20260809 02:02:57.332192 9041 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/master-1/data/info.pb
I20260809 02:02:57.333770 9234 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.253:40841 every 8 connection(s)
I20260809 02:02:57.334240 9235 sys_catalog.cc:263] Verifying existing consensus state
I20260809 02:02:57.335247 9235 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf6ddc1ba97745d897049f1300717b5b: Bootstrap starting.
I20260809 02:02:57.346402 9235 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf6ddc1ba97745d897049f1300717b5b: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 02:02:57.346912 9235 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf6ddc1ba97745d897049f1300717b5b: Bootstrap complete.
I20260809 02:02:57.347920 9235 raft_consensus.cc:348] T 00000000000000000000000000000000 P bf6ddc1ba97745d897049f1300717b5b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "14381b53c95b4bda8c4273431f4a1762" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 34543 } } peers { permanent_uuid: "bf6ddc1ba97745d897049f1300717b5b" member_type: NON_VOTER last_known_addr { host: "127.1.192.253" port: 40841 } attrs { promote: true } }
I20260809 02:02:57.348068 9235 raft_consensus.cc:729] T 00000000000000000000000000000000 P bf6ddc1ba97745d897049f1300717b5b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: bf6ddc1ba97745d897049f1300717b5b, State: Initialized, Role: LEARNER
I20260809 02:02:57.348311 9235 consensus_queue.cc:260] T 00000000000000000000000000000000 P bf6ddc1ba97745d897049f1300717b5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "14381b53c95b4bda8c4273431f4a1762" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 34543 } } peers { permanent_uuid: "bf6ddc1ba97745d897049f1300717b5b" member_type: NON_VOTER last_known_addr { host: "127.1.192.253" port: 40841 } attrs { promote: true } }
I20260809 02:02:57.348802 9238 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf6ddc1ba97745d897049f1300717b5b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "14381b53c95b4bda8c4273431f4a1762" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 34543 } } peers { permanent_uuid: "bf6ddc1ba97745d897049f1300717b5b" member_type: NON_VOTER last_known_addr { host: "127.1.192.253" port: 40841 } attrs { promote: true } } }
I20260809 02:02:57.348909 9235 sys_catalog.cc:564] T 00000000000000000000000000000000 P bf6ddc1ba97745d897049f1300717b5b [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:02:57.348999 9238 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf6ddc1ba97745d897049f1300717b5b [sys.catalog]: This master's current role is: LEARNER
I20260809 02:02:57.352445 9250 catalog_manager.cc:1259] Loaded cluster ID: 279e275c39804b08b8c01de18b06bcb9
I20260809 02:02:57.352648 9250 catalog_manager.cc:1552] T 00000000000000000000000000000000 P bf6ddc1ba97745d897049f1300717b5b: loading cluster ID for follower catalog manager: success
I20260809 02:02:57.354984 9250 catalog_manager.cc:1574] T 00000000000000000000000000000000 P bf6ddc1ba97745d897049f1300717b5b: acquiring CA information for follower catalog manager: success
I20260809 02:02:57.356699 9250 catalog_manager.cc:1602] T 00000000000000000000000000000000 P bf6ddc1ba97745d897049f1300717b5b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260809 02:02:57.399475 9159 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:02:57.399799 9159 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:02:57.399884 9159 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:02:57.404070 9159 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:02:57.404188 9159 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:02:57.404223 9159 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:02:57.404266 9159 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:02:57.404305 9159 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:02:57.410024 9159 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:44161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.1.192.254:34543,127.1.192.253:40841,127.1.192.252:44581
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/krb5kdc/oryx_127.1.192.252.keytab
--principal=oryx/127.1.192.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.192.252:44581
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.1.192.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9159
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:02:57.411190 9159 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:02:57.412237 9159 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:02:57.421286 9258 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:02:57.421370 9256 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:02:57.421351 9255 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:02:57.421588 9159 server_base.cc:1034] running on GCE node
I20260809 02:02:57.422246 9159 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:02:57.423435 9159 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:02:57.424633 9159 hybrid_clock.cc:648] HybridClock initialized: now 1786240977424616 us; error 32 us; skew 500 ppm
Aug 09 02:02:57 dist-test-slave-ss4n krb5kdc[8478](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240977, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.192.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:02:57.427561 9159 init.cc:377] Logged in from keytab as oryx/127.1.192.252@KRBTEST.COM (short username oryx)
I20260809 02:02:57.428491 9159 webserver.cc:460] Webserver started at http://127.1.192.252:39053/ using document root <none> and password file <none>
I20260809 02:02:57.429013 9159 fs_manager.cc:359] Metadata directory not provided
I20260809 02:02:57.429071 9159 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:02:57.429301 9159 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:02:57.431399 9159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/master-2/data/instance:
uuid: "d909896773e04b93bbb1e1e7d4c11264"
format_stamp: "Formatted at 2026-08-09 02:02:57 on dist-test-slave-ss4n"
server_key: "980b5f4100ccdbb6451d0aac9e375e6c"
server_key_iv: "bcd2375561df2b7188e27e61eb1038b6"
server_key_version: "encryptionkey@0"
I20260809 02:02:57.431963 9159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/master-2/wal/instance:
uuid: "d909896773e04b93bbb1e1e7d4c11264"
format_stamp: "Formatted at 2026-08-09 02:02:57 on dist-test-slave-ss4n"
server_key: "980b5f4100ccdbb6451d0aac9e375e6c"
server_key_iv: "bcd2375561df2b7188e27e61eb1038b6"
server_key_version: "encryptionkey@0"
I20260809 02:02:57.436007 9159 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 02:02:57.438614 9266 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:57.439998 9159 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 02:02:57.440148 9159 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/master-2/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/master-2/wal
uuid: "d909896773e04b93bbb1e1e7d4c11264"
format_stamp: "Formatted at 2026-08-09 02:02:57 on dist-test-slave-ss4n"
server_key: "980b5f4100ccdbb6451d0aac9e375e6c"
server_key_iv: "bcd2375561df2b7188e27e61eb1038b6"
server_key_version: "encryptionkey@0"
I20260809 02:02:57.440270 9159 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:02:57.441242 8738 heartbeater.cc:503] Master 127.1.192.254:34543 was elected leader, sending a full tablet report...
I20260809 02:02:57.463989 9159 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:02:57.467628 9159 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:02:57.467855 9159 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:02:57.476913 9159 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.252:44581
I20260809 02:02:57.477242 9325 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.252:44581 every 8 connection(s)
I20260809 02:02:57.478132 9159 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/master-2/data/info.pb
I20260809 02:02:57.480295 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 9159
I20260809 02:02:57.481307 9326 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:02:57.485707 9326 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.192.254" port: 34543 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 02:02:57 dist-test-slave-ss4n krb5kdc[8478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240975, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.192.252@KRBTEST.COM
Aug 09 02:02:57 dist-test-slave-ss4n krb5kdc[8478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240977, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.192.252@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM
I20260809 02:02:57.497067 9328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:57.481898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32786 (local address 127.1.192.252:44581)
0809 02:02:57.482374 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:57.482390 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:57.482428 (+ 38us) server_negotiation.cc:407] Connection header received
0809 02:02:57.483295 (+ 867us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:57.483313 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:57.483653 (+ 340us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:57.484014 (+ 361us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:57.484933 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:57.486158 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:57.487128 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:57.487703 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:57.488345 (+ 642us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:57.491327 (+ 2982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:02:57.491332 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:02:57.491341 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:02:57.491369 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:02:57.493407 (+ 2038us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:57.494038 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:57.494044 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:57.494052 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:57.494193 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:57.494514 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:57.494517 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:57.494519 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:57.495131 (+ 612us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:02:57.495353 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:57.495704 (+ 351us) server_negotiation.cc:299] Negotiation successful
0809 02:02:57.496045 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":330,"spinlock_wait_cycles":768,"thread_start_us":166,"threads_started":1}
I20260809 02:02:57.498152 8572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:57.487724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60468 (local address 127.1.192.254:34543)
0809 02:02:57.487877 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:57.487882 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:57.488154 (+ 272us) server_negotiation.cc:407] Connection header received
0809 02:02:57.488332 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:57.488342 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:57.488392 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:57.488474 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:57.489718 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:57.490417 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:57.491474 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:57.491711 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:57.491834 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:57.494338 (+ 2504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:02:57.494341 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:02:57.494343 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:02:57.494376 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:02:57.496045 (+ 1669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:57.496479 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:57.496483 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:57.496484 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:57.496577 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:57.496973 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:57.496977 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:57.496979 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:57.497293 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:02:57.497413 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:57.497853 (+ 440us) server_negotiation.cc:299] Negotiation successful
0809 02:02:57.497977 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 02:02:57.498162 9329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:57.487544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:34543 (local address 127.0.0.1:60468)
0809 02:02:57.487989 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:57.488046 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0809 02:02:57.488181 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:02:57.488729 (+ 548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:02:57.488744 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:02:57.489250 (+ 506us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:02:57.489517 (+ 267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:57.489530 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:57.490605 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:57.490608 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:57.491336 (+ 728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:57.491345 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:57.491836 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:57.491840 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:57.491958 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:57.492501 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:02:57.492512 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:02:57.494184 (+ 1672us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:02:57.496156 (+ 1972us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:57.496164 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:57.496168 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:57.496381 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:57.496702 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:57.496704 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:57.496706 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:57.496820 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:57.497412 (+ 592us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:02:57.497419 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:02:57.497634 (+ 215us) client_negotiation.cc:769] Sending connection context
0809 02:02:57.497784 (+ 150us) client_negotiation.cc:240] Negotiation successful
0809 02:02:57.497977 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":178,"threads_started":1}
I20260809 02:02:57.499290 9326 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.192.253" port: 40841 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 02:02:57 dist-test-slave-ss4n krb5kdc[8478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240977, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.192.252@KRBTEST.COM for oryx/127.1.192.253@KRBTEST.COM
I20260809 02:02:57.508075 9329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:57.499796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.253:40841 (local address 127.0.0.1:48564)
0809 02:02:57.500134 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:57.500148 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 02:02:57.500210 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:02:57.500422 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:02:57.500424 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:02:57.500558 (+ 134us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:02:57.500661 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:57.500666 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:57.501876 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:57.501879 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:57.502417 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:57.502425 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:57.502948 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:57.502950 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:57.503025 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:57.503461 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:02:57.503471 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:02:57.504767 (+ 1296us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:02:57.506620 (+ 1853us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:57.506624 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:57.506626 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:57.506868 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:57.507195 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:57.507197 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:57.507198 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:57.507272 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:57.507676 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:02:57.507678 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:02:57.507758 (+ 80us) client_negotiation.cc:769] Sending connection context
0809 02:02:57.507841 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 02:02:57.507951 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277}
I20260809 02:02:57.508121 9330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:57.499932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48564 (local address 127.1.192.253:40841)
0809 02:02:57.500201 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:57.500204 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:57.500218 (+ 14us) server_negotiation.cc:407] Connection header received
0809 02:02:57.500276 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:57.500278 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:57.500318 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:57.500390 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:57.500766 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:57.501448 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:57.502580 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:57.502838 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:57.502945 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:57.504902 (+ 1957us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:02:57.504905 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:02:57.504907 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:02:57.504939 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:02:57.506513 (+ 1574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:57.507021 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:57.507024 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:57.507025 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:57.507115 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:57.507348 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:57.507350 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:57.507351 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:57.507520 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:02:57.507665 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:57.507897 (+ 232us) server_negotiation.cc:299] Negotiation successful
0809 02:02:57.507996 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":194,"thread_start_us":83,"threads_started":1}
I20260809 02:02:57.509097 9326 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.192.252" port: 44581 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 02:02:57 dist-test-slave-ss4n krb5kdc[8478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240977, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.192.252@KRBTEST.COM for oryx/127.1.192.252@KRBTEST.COM
I20260809 02:02:57.517068 9329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:57.509702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.252:44581 (local address 127.0.0.1:32802)
0809 02:02:57.509825 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:57.509841 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 02:02:57.509916 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:02:57.510101 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:02:57.510103 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:02:57.510232 (+ 129us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:02:57.510305 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:57.510309 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:57.511410 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:57.511413 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:57.511913 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:57.511922 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:57.512333 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:57.512335 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:57.512394 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:57.512920 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:02:57.512931 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:02:57.514297 (+ 1366us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:02:57.515800 (+ 1503us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:57.515803 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:57.515804 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:57.515988 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:57.516235 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:57.516237 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:57.516238 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:57.516313 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:57.516653 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:02:57.516655 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:02:57.516730 (+ 75us) client_negotiation.cc:769] Sending connection context
0809 02:02:57.516808 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 02:02:57.516919 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260809 02:02:57.517095 9328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:57.509776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32802 (local address 127.1.192.252:44581)
0809 02:02:57.509895 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:57.509898 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:57.509910 (+ 12us) server_negotiation.cc:407] Connection header received
0809 02:02:57.509974 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:57.509976 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:57.510009 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:57.510071 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:57.510379 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:57.510893 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:57.512019 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:57.512236 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:57.512314 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:57.514422 (+ 2108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:02:57.514425 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:02:57.514427 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:02:57.514448 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:02:57.515692 (+ 1244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:57.516077 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:57.516080 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:57.516081 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:57.516152 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:57.516390 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:57.516392 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:57.516393 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:57.516555 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:02:57.516629 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:57.516844 (+ 215us) server_negotiation.cc:299] Negotiation successful
0809 02:02:57.516975 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":34,"server-negotiator.queue_time_us":49}
I20260809 02:02:57.520177 9326 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d909896773e04b93bbb1e1e7d4c11264: Bootstrap starting.
I20260809 02:02:57.522491 9326 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d909896773e04b93bbb1e1e7d4c11264: Neither blocks nor log segments found. Creating new log.
I20260809 02:02:57.523264 9326 log.cc:824] T 00000000000000000000000000000000 P d909896773e04b93bbb1e1e7d4c11264: Log is configured to *not* fsync() on all Append() calls
I20260809 02:02:57.525279 9326 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d909896773e04b93bbb1e1e7d4c11264: No bootstrap required, opened a new log
I20260809 02:02:57.528405 9326 raft_consensus.cc:348] T 00000000000000000000000000000000 P d909896773e04b93bbb1e1e7d4c11264 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14381b53c95b4bda8c4273431f4a1762" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 34543 } } peers { permanent_uuid: "bf6ddc1ba97745d897049f1300717b5b" member_type: VOTER last_known_addr { host: "127.1.192.253" port: 40841 } } peers { permanent_uuid: "d909896773e04b93bbb1e1e7d4c11264" member_type: VOTER last_known_addr { host: "127.1.192.252" port: 44581 } }
I20260809 02:02:57.528609 9326 raft_consensus.cc:374] T 00000000000000000000000000000000 P d909896773e04b93bbb1e1e7d4c11264 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:02:57.528703 9326 raft_consensus.cc:729] T 00000000000000000000000000000000 P d909896773e04b93bbb1e1e7d4c11264 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d909896773e04b93bbb1e1e7d4c11264, State: Initialized, Role: FOLLOWER
I20260809 02:02:57.529115 9326 consensus_queue.cc:260] T 00000000000000000000000000000000 P d909896773e04b93bbb1e1e7d4c11264 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14381b53c95b4bda8c4273431f4a1762" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 34543 } } peers { permanent_uuid: "bf6ddc1ba97745d897049f1300717b5b" member_type: VOTER last_known_addr { host: "127.1.192.253" port: 40841 } } peers { permanent_uuid: "d909896773e04b93bbb1e1e7d4c11264" member_type: VOTER last_known_addr { host: "127.1.192.252" port: 44581 } }
I20260809 02:02:57.529659 9332 sys_catalog.cc:455] T 00000000000000000000000000000000 P d909896773e04b93bbb1e1e7d4c11264 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14381b53c95b4bda8c4273431f4a1762" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 34543 } } peers { permanent_uuid: "bf6ddc1ba97745d897049f1300717b5b" member_type: VOTER last_known_addr { host: "127.1.192.253" port: 40841 } } peers { permanent_uuid: "d909896773e04b93bbb1e1e7d4c11264" member_type: VOTER last_known_addr { host: "127.1.192.252" port: 44581 } } }
I20260809 02:02:57.529810 9332 sys_catalog.cc:458] T 00000000000000000000000000000000 P d909896773e04b93bbb1e1e7d4c11264 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 02:02:57.530395 9326 sys_catalog.cc:564] T 00000000000000000000000000000000 P d909896773e04b93bbb1e1e7d4c11264 [sys.catalog]: configured and running, proceeding with master startup.
W20260809 02:02:57.534976 9343 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d909896773e04b93bbb1e1e7d4c11264: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 02:02:57.535080 9343 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 02:02:57.538515 9159 master_runner.cc:428] Detected that this master d909896773e04b93bbb1e1e7d4c11264 is joining an existing cluster
I20260809 02:02:57.538570 9159 master_runner.cc:432] Initiating AddMaster RPC to add 127.1.192.252:44581
I20260809 02:02:57.550382 9351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:57.541059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.253:40841 (local address 127.0.0.1:48580)
0809 02:02:57.541615 (+ 556us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:57.541627 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:02:57.541761 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:02:57.542134 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:02:57.542137 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:02:57.542338 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:02:57.542470 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:57.542476 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:57.543511 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:57.543514 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:57.545018 (+ 1504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:57.545027 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:57.545773 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:57.545776 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:57.545863 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:57.546365 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:02:57.546384 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:02:57.546724 (+ 340us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:02:57.548605 (+ 1881us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:57.548608 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:57.548609 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:57.548806 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:57.549115 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:57.549117 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:57.549119 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:57.549210 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:57.549748 (+ 538us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:02:57.549752 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:02:57.549854 (+ 102us) client_negotiation.cc:769] Sending connection context
0809 02:02:57.549973 (+ 119us) client_negotiation.cc:240] Negotiation successful
0809 02:02:57.550173 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"mutex_wait_us":29,"thread_start_us":96,"threads_started":1}
I20260809 02:02:57.550396 9330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:57.541127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48580 (local address 127.1.192.253:40841)
0809 02:02:57.541269 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:57.541274 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:57.541702 (+ 428us) server_negotiation.cc:407] Connection header received
0809 02:02:57.541868 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:57.541871 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:57.541928 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:57.541981 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:57.542596 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:57.543384 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:57.545145 (+ 1761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:57.545532 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:57.545608 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:57.546869 (+ 1261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:02:57.546873 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:02:57.546875 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:02:57.546902 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:02:57.548451 (+ 1549us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:57.548909 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:57.548913 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:57.548915 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:57.549015 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:57.549314 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:57.549317 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:57.549319 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:57.549632 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:02:57.549744 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:57.550059 (+ 315us) server_negotiation.cc:299] Negotiation successful
0809 02:02:57.550228 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 02:02:57.550573 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 8595
I20260809 02:02:57.564551 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.192.193:37543
--local_ip_for_outbound_sockets=127.1.192.193
--tserver_master_addrs=127.1.192.254:34543
--webserver_port=35199
--webserver_interface=127.1.192.193
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/krb5kdc/oryx_127.1.192.193.keytab
--principal=oryx/127.1.192.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:44161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.1.192.254:34543,127.1.192.253:40841
--tserver_master_addrs=127.1.192.254:34543,127.1.192.253:40841,127.1.192.252:44581 with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 02:02:57.569824 9349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:57.540794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.252:44581 (local address 127.0.0.1:32812)
0809 02:02:57.541367 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:57.541381 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 02:02:57.541529 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:02:57.541841 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:02:57.541844 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:02:57.542161 (+ 317us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:02:57.542304 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:57.542312 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:57.543336 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:57.543339 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:57.544287 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:57.544298 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:57.545058 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:57.545061 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:57.545175 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:57.545820 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:02:57.545839 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:02:57.565959 (+ 20120us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:02:57.568084 (+ 2125us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:57.568087 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:57.568089 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:57.568326 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:57.568567 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:57.568570 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:57.568572 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:57.568641 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:57.569171 (+ 530us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:02:57.569173 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:02:57.569260 (+ 87us) client_negotiation.cc:769] Sending connection context
0809 02:02:57.569354 (+ 94us) client_negotiation.cc:240] Negotiation successful
0809 02:02:57.569528 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":508,"thread_start_us":103,"threads_started":1}
I20260809 02:02:57.569963 9328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:57.540677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32812 (local address 127.1.192.252:44581)
0809 02:02:57.540816 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:57.540822 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:57.541449 (+ 627us) server_negotiation.cc:407] Connection header received
0809 02:02:57.541627 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:57.541630 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:57.541668 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:57.541715 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:57.542426 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:57.543210 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:57.544412 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:57.544759 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:57.544834 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:57.566173 (+ 21339us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:02:57.566178 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:02:57.566180 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:02:57.566206 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:02:57.567961 (+ 1755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:57.568419 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:57.568421 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:57.568423 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:57.568486 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:57.568727 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:57.568730 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:57.568731 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:57.569006 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:02:57.569137 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:57.569686 (+ 549us) server_negotiation.cc:299] Negotiation successful
0809 02:02:57.569785 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 02:02:57.571512 9350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:57.540605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:34543 (local address 127.0.0.1:60482)
0809 02:02:57.541174 (+ 569us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:57.541189 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 02:02:57.541333 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:02:57.541760 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:02:57.541762 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:02:57.542087 (+ 325us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:02:57.542259 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:57.542268 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:57.543419 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:57.543423 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:57.544616 (+ 1193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:57.544626 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:57.545472 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:57.545525 (+ 53us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:57.545622 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:57.566069 (+ 20447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:02:57.566094 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:02:57.566682 (+ 588us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:02:57.568904 (+ 2222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:57.568907 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:57.568909 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:57.569221 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:57.569583 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:57.569589 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:57.569593 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:57.570073 (+ 480us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:57.570814 (+ 741us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:02:57.570820 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:02:57.570945 (+ 125us) client_negotiation.cc:769] Sending connection context
0809 02:02:57.571066 (+ 121us) client_negotiation.cc:240] Negotiation successful
0809 02:02:57.571237 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":486,"mutex_wait_us":32,"thread_start_us":108,"threads_started":1}
I20260809 02:02:57.572402 8572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:57.540688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60482 (local address 127.1.192.254:34543)
0809 02:02:57.540930 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:57.540935 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:57.541260 (+ 325us) server_negotiation.cc:407] Connection header received
0809 02:02:57.541456 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:57.541461 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:57.541545 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:57.541623 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:57.542405 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:57.543276 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:57.544751 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:57.545113 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:57.545201 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:57.566905 (+ 21704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:02:57.566912 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:02:57.566919 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:02:57.566954 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:02:57.568747 (+ 1793us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:57.569348 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:57.569354 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:57.569359 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:57.569449 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:57.570466 (+ 1017us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:57.570470 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:57.570472 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:57.570699 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:02:57.570804 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:57.571986 (+ 1182us) server_negotiation.cc:299] Negotiation successful
0809 02:02:57.572154 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152}
Aug 09 02:02:57 dist-test-slave-ss4n krb5kdc[8478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240976, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.192.254@KRBTEST.COM for oryx/127.1.192.252@KRBTEST.COM
I20260809 02:02:57.587297 9147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:57.576608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.252:44581 (local address 127.0.0.1:32828)
0809 02:02:57.576742 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:57.576757 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 02:02:57.576855 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:02:57.578041 (+ 1186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:02:57.578044 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:02:57.578269 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:02:57.578406 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:57.578414 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:57.579458 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:57.579461 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:57.580325 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:57.580339 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:57.581007 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:57.581011 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:57.581118 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:57.581732 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:02:57.581747 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:02:57.583369 (+ 1622us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:02:57.585379 (+ 2010us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:57.585383 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:57.585385 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:57.585775 (+ 390us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:57.586124 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:57.586128 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:57.586130 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:57.586220 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:57.586742 (+ 522us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:02:57.586745 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:02:57.586847 (+ 102us) client_negotiation.cc:769] Sending connection context
0809 02:02:57.586945 (+ 98us) client_negotiation.cc:240] Negotiation successful
0809 02:02:57.587094 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260809 02:02:57.587908 9328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:57.576664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32828 (local address 127.1.192.252:44581)
0809 02:02:57.576768 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:57.576774 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:57.576808 (+ 34us) server_negotiation.cc:407] Connection header received
0809 02:02:57.577778 (+ 970us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:57.577782 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:57.577838 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:57.577918 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:57.578539 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:57.579314 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:57.580469 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:57.580797 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:57.580890 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:57.583528 (+ 2638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:02:57.583532 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:02:57.583535 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:02:57.583562 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:02:57.585250 (+ 1688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:57.585890 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:57.585894 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:57.585896 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:57.586000 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:57.586336 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:57.586340 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:57.586342 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:57.586602 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:02:57.586696 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:57.587643 (+ 947us) server_negotiation.cc:299] Negotiation successful
0809 02:02:57.587762 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 02:02:57.588948 8527 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.1.192.252:44581: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "d909896773e04b93bbb1e1e7d4c11264"
member_type: NON_VOTER
last_known_addr {
host: "127.1.192.252"
port: 44581
}
attrs {
promote: true
}
}
dest_uuid: "14381b53c95b4bda8c4273431f4a1762"
cas_config_opid_index: 5
I20260809 02:02:57.589617 8527 consensus_queue.cc:237] T 00000000000000000000000000000000 P 14381b53c95b4bda8c4273431f4a1762 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "14381b53c95b4bda8c4273431f4a1762" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 34543 } } peers { permanent_uuid: "bf6ddc1ba97745d897049f1300717b5b" member_type: NON_VOTER last_known_addr { host: "127.1.192.253" port: 40841 } attrs { promote: true } } peers { permanent_uuid: "d909896773e04b93bbb1e1e7d4c11264" member_type: NON_VOTER last_known_addr { host: "127.1.192.252" port: 44581 } attrs { promote: true } }
I20260809 02:02:57.590799 9153 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 14381b53c95b4bda8c4273431f4a1762 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER d909896773e04b93bbb1e1e7d4c11264 (127.1.192.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "14381b53c95b4bda8c4273431f4a1762" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 34543 } } peers { permanent_uuid: "bf6ddc1ba97745d897049f1300717b5b" member_type: NON_VOTER last_known_addr { host: "127.1.192.253" port: 40841 } attrs { promote: true } } peers { permanent_uuid: "d909896773e04b93bbb1e1e7d4c11264" member_type: NON_VOTER last_known_addr { host: "127.1.192.252" port: 44581 } attrs { promote: true } } }
I20260809 02:02:57.591239 9299 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d909896773e04b93bbb1e1e7d4c11264 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:02:57.592329 9153 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.1.192.252:44581
I20260809 02:02:57.592501 9153 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14381b53c95b4bda8c4273431f4a1762 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "14381b53c95b4bda8c4273431f4a1762" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "14381b53c95b4bda8c4273431f4a1762" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 34543 } } peers { permanent_uuid: "bf6ddc1ba97745d897049f1300717b5b" member_type: NON_VOTER last_known_addr { host: "127.1.192.253" port: 40841 } attrs { promote: true } } peers { permanent_uuid: "d909896773e04b93bbb1e1e7d4c11264" member_type: NON_VOTER last_known_addr { host: "127.1.192.252" port: 44581 } attrs { promote: true } } }
I20260809 02:02:57.592589 9153 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14381b53c95b4bda8c4273431f4a1762 [sys.catalog]: This master's current role is: LEADER
I20260809 02:02:57.592646 9299 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d909896773e04b93bbb1e1e7d4c11264 [term 1 FOLLOWER]: Refusing update from remote peer 14381b53c95b4bda8c4273431f4a1762: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260809 02:02:57.593254 9152 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 14381b53c95b4bda8c4273431f4a1762 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d909896773e04b93bbb1e1e7d4c11264" member_type: NON_VOTER last_known_addr { host: "127.1.192.252" port: 44581 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:02:57.593787 9159 master.cc:537] Master@127.1.192.252:44581 shutting down...
W20260809 02:02:57.595160 8511 consensus_peers.cc:487] T 00000000000000000000000000000000 P 14381b53c95b4bda8c4273431f4a1762 -> Peer d909896773e04b93bbb1e1e7d4c11264 (127.1.192.252:44581): Couldn't send request to peer d909896773e04b93bbb1e1e7d4c11264. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260809 02:02:57.598989 9159 raft_consensus.cc:2229] T 00000000000000000000000000000000 P d909896773e04b93bbb1e1e7d4c11264 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:02:57.599324 9159 raft_consensus.cc:2258] T 00000000000000000000000000000000 P d909896773e04b93bbb1e1e7d4c11264 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:02:57.599445 9159 tablet_replica.cc:330] T 00000000000000000000000000000000 P d909896773e04b93bbb1e1e7d4c11264: stopping tablet replica
I20260809 02:02:57.600324 9147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:57.590867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.253:40841 (local address 127.0.0.1:48590)
0809 02:02:57.591119 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:57.591134 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 02:02:57.591213 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:02:57.595563 (+ 4350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:02:57.595569 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:02:57.595604 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:02:57.595703 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:57.595708 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:57.596745 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:57.596751 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:57.598255 (+ 1504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:57.598269 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:57.599838 (+ 1569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:57.599841 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:57.599948 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:57.600007 (+ 59us) client_negotiation.cc:769] Sending connection context
0809 02:02:57.600095 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 02:02:57.600191 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176}
I20260809 02:02:57.600355 9330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:57.592764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48590 (local address 127.1.192.253:40841)
0809 02:02:57.595251 (+ 2487us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:57.595259 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:57.595277 (+ 18us) server_negotiation.cc:407] Connection header received
0809 02:02:57.595339 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:57.595343 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:57.595415 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:57.595528 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:02:57.595903 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:57.596623 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:57.598414 (+ 1791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:57.599427 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:57.599535 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:57.599684 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:57.600143 (+ 459us) server_negotiation.cc:299] Negotiation successful
0809 02:02:57.600230 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2362}
I20260809 02:02:57.601126 9205 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bf6ddc1ba97745d897049f1300717b5b [term 1 LEARNER]: Refusing update from remote peer 14381b53c95b4bda8c4273431f4a1762: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260809 02:02:57.601648 9152 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 14381b53c95b4bda8c4273431f4a1762 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf6ddc1ba97745d897049f1300717b5b" member_type: NON_VOTER last_known_addr { host: "127.1.192.253" port: 40841 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 02:02:57.603667 9205 raft_consensus.cc:2932] T 00000000000000000000000000000000 P bf6ddc1ba97745d897049f1300717b5b [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER d909896773e04b93bbb1e1e7d4c11264 (127.1.192.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "14381b53c95b4bda8c4273431f4a1762" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 34543 } } peers { permanent_uuid: "bf6ddc1ba97745d897049f1300717b5b" member_type: NON_VOTER last_known_addr { host: "127.1.192.253" port: 40841 } attrs { promote: true } } peers { permanent_uuid: "d909896773e04b93bbb1e1e7d4c11264" member_type: NON_VOTER last_known_addr { host: "127.1.192.252" port: 44581 } attrs { promote: true } } }
I20260809 02:02:57.605564 9238 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf6ddc1ba97745d897049f1300717b5b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 14381b53c95b4bda8c4273431f4a1762. Latest consensus state: current_term: 1 leader_uuid: "14381b53c95b4bda8c4273431f4a1762" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "14381b53c95b4bda8c4273431f4a1762" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 34543 } } peers { permanent_uuid: "bf6ddc1ba97745d897049f1300717b5b" member_type: NON_VOTER last_known_addr { host: "127.1.192.253" port: 40841 } attrs { promote: true } } peers { permanent_uuid: "d909896773e04b93bbb1e1e7d4c11264" member_type: NON_VOTER last_known_addr { host: "127.1.192.252" port: 44581 } attrs { promote: true } } }
I20260809 02:02:57.605682 9238 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf6ddc1ba97745d897049f1300717b5b [sys.catalog]: This master's current role is: LEARNER
I20260809 02:02:57.605757 9238 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf6ddc1ba97745d897049f1300717b5b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "14381b53c95b4bda8c4273431f4a1762" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "14381b53c95b4bda8c4273431f4a1762" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 34543 } } peers { permanent_uuid: "bf6ddc1ba97745d897049f1300717b5b" member_type: NON_VOTER last_known_addr { host: "127.1.192.253" port: 40841 } attrs { promote: true } } peers { permanent_uuid: "d909896773e04b93bbb1e1e7d4c11264" member_type: NON_VOTER last_known_addr { host: "127.1.192.252" port: 44581 } attrs { promote: true } } }
I20260809 02:02:57.605831 9238 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf6ddc1ba97745d897049f1300717b5b [sys.catalog]: This master's current role is: LEARNER
I20260809 02:02:57.608340 9153 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 14381b53c95b4bda8c4273431f4a1762: attempting to promote NON_VOTER bf6ddc1ba97745d897049f1300717b5b to VOTER
I20260809 02:02:57.608830 9153 consensus_queue.cc:237] T 00000000000000000000000000000000 P 14381b53c95b4bda8c4273431f4a1762 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "14381b53c95b4bda8c4273431f4a1762" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 34543 } } peers { permanent_uuid: "bf6ddc1ba97745d897049f1300717b5b" member_type: VOTER last_known_addr { host: "127.1.192.253" port: 40841 } attrs { promote: false } } peers { permanent_uuid: "d909896773e04b93bbb1e1e7d4c11264" member_type: NON_VOTER last_known_addr { host: "127.1.192.252" port: 44581 } attrs { promote: true } }
I20260809 02:02:57.610081 9147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:57.609792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.252:44581 (local address 127.0.0.1:32838)
0809 02:02:57.609918 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:57.609996 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.252:44581: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260809 02:02:57.610159 9205 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bf6ddc1ba97745d897049f1300717b5b [term 1 LEARNER]: Refusing update from remote peer 14381b53c95b4bda8c4273431f4a1762: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260809 02:02:57.610519 9152 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 14381b53c95b4bda8c4273431f4a1762 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf6ddc1ba97745d897049f1300717b5b" member_type: VOTER last_known_addr { host: "127.1.192.253" port: 40841 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260809 02:02:57.610898 9147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:57.610591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.252:44581 (local address 127.0.0.1:32844)
0809 02:02:57.610784 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:57.610835 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.252:44581: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126}
W20260809 02:02:57.611085 8511 consensus_peers.cc:487] T 00000000000000000000000000000000 P 14381b53c95b4bda8c4273431f4a1762 -> Peer d909896773e04b93bbb1e1e7d4c11264 (127.1.192.252:44581): Couldn't send request to peer d909896773e04b93bbb1e1e7d4c11264. Status: Network error: Client connection negotiation failed: client connection to 127.1.192.252:44581: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260809 02:02:57.612255 9153 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 14381b53c95b4bda8c4273431f4a1762 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, bf6ddc1ba97745d897049f1300717b5b (127.1.192.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "14381b53c95b4bda8c4273431f4a1762" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 34543 } } peers { permanent_uuid: "bf6ddc1ba97745d897049f1300717b5b" member_type: VOTER last_known_addr { host: "127.1.192.253" port: 40841 } attrs { promote: false } } peers { permanent_uuid: "d909896773e04b93bbb1e1e7d4c11264" member_type: NON_VOTER last_known_addr { host: "127.1.192.252" port: 44581 } attrs { promote: true } } }
I20260809 02:02:57.612635 9204 raft_consensus.cc:2932] T 00000000000000000000000000000000 P bf6ddc1ba97745d897049f1300717b5b [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, bf6ddc1ba97745d897049f1300717b5b (127.1.192.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "14381b53c95b4bda8c4273431f4a1762" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 34543 } } peers { permanent_uuid: "bf6ddc1ba97745d897049f1300717b5b" member_type: VOTER last_known_addr { host: "127.1.192.253" port: 40841 } attrs { promote: false } } peers { permanent_uuid: "d909896773e04b93bbb1e1e7d4c11264" member_type: NON_VOTER last_known_addr { host: "127.1.192.252" port: 44581 } attrs { promote: true } } }
I20260809 02:02:57.613970 9153 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14381b53c95b4bda8c4273431f4a1762 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "14381b53c95b4bda8c4273431f4a1762" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "14381b53c95b4bda8c4273431f4a1762" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 34543 } } peers { permanent_uuid: "bf6ddc1ba97745d897049f1300717b5b" member_type: VOTER last_known_addr { host: "127.1.192.253" port: 40841 } attrs { promote: false } } peers { permanent_uuid: "d909896773e04b93bbb1e1e7d4c11264" member_type: NON_VOTER last_known_addr { host: "127.1.192.252" port: 44581 } attrs { promote: true } } }
I20260809 02:02:57.614073 9153 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14381b53c95b4bda8c4273431f4a1762 [sys.catalog]: This master's current role is: LEADER
I20260809 02:02:57.614406 9151 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14381b53c95b4bda8c4273431f4a1762 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "14381b53c95b4bda8c4273431f4a1762" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "14381b53c95b4bda8c4273431f4a1762" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 34543 } } peers { permanent_uuid: "bf6ddc1ba97745d897049f1300717b5b" member_type: VOTER last_known_addr { host: "127.1.192.253" port: 40841 } attrs { promote: false } } peers { permanent_uuid: "d909896773e04b93bbb1e1e7d4c11264" member_type: NON_VOTER last_known_addr { host: "127.1.192.252" port: 44581 } attrs { promote: true } } }
I20260809 02:02:57.614528 9151 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14381b53c95b4bda8c4273431f4a1762 [sys.catalog]: This master's current role is: LEADER
I20260809 02:02:57.614763 9354 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf6ddc1ba97745d897049f1300717b5b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "14381b53c95b4bda8c4273431f4a1762" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "14381b53c95b4bda8c4273431f4a1762" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 34543 } } peers { permanent_uuid: "bf6ddc1ba97745d897049f1300717b5b" member_type: VOTER last_known_addr { host: "127.1.192.253" port: 40841 } attrs { promote: false } } peers { permanent_uuid: "d909896773e04b93bbb1e1e7d4c11264" member_type: NON_VOTER last_known_addr { host: "127.1.192.252" port: 44581 } attrs { promote: true } } }
I20260809 02:02:57.614939 9354 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf6ddc1ba97745d897049f1300717b5b [sys.catalog]: This master's current role is: FOLLOWER
I20260809 02:02:57.618930 9159 master.cc:552] Master@127.1.192.252:44581 shutdown complete.
I20260809 02:02:57.618994 9159 master_runner.cc:305] Clearing existing system tablet
I20260809 02:02:57.619251 9159 fs_manager.cc:359] Metadata directory not provided
I20260809 02:02:57.619309 9159 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:02:57.620666 9159 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 02:02:57.621636 9356 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:57.621867 9159 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:02:57.622054 9159 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:02:57.622125 9159 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/master-2/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/master-2/wal
uuid: "d909896773e04b93bbb1e1e7d4c11264"
format_stamp: "Formatted at 2026-08-09 02:02:57 on dist-test-slave-ss4n"
server_key: "980b5f4100ccdbb6451d0aac9e375e6c"
server_key_iv: "bcd2375561df2b7188e27e61eb1038b6"
server_key_version: "encryptionkey@0"
I20260809 02:02:57.624212 9159 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P d909896773e04b93bbb1e1e7d4c11264: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:02:57.626667 9159 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P d909896773e04b93bbb1e1e7d4c11264: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260809 02:02:57.626801 9159 log.cc:1196] T 00000000000000000000000000000000 P d909896773e04b93bbb1e1e7d4c11264: Deleting WAL directory at /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260809 02:02:57.627105 9159 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P d909896773e04b93bbb1e1e7d4c11264: Deleting consensus metadata
I20260809 02:02:57.627341 9159 master_runner.cc:315] Copying system tablet from 127.1.192.254:34543
I20260809 02:02:57.628577 9159 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P d909896773e04b93bbb1e1e7d4c11264: tablet copy: Beginning tablet copy session from remote peer at address 127.1.192.254:34543
I20260809 02:02:57.643110 8572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:57.634034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60486 (local address 127.1.192.254:34543)
0809 02:02:57.634190 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:57.634195 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:57.634503 (+ 308us) server_negotiation.cc:407] Connection header received
0809 02:02:57.634654 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:57.634658 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:57.634711 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:57.634764 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:57.635461 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:57.636283 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:57.637570 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:57.637896 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:57.638012 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:57.639406 (+ 1394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:02:57.639410 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:02:57.639413 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:02:57.639441 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:02:57.641196 (+ 1755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:57.641733 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:57.641737 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:57.641739 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:57.641835 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:57.642151 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:57.642154 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:57.642156 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:57.642397 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:02:57.642500 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:57.642814 (+ 314us) server_negotiation.cc:299] Negotiation successful
0809 02:02:57.642949 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 02:02:57.643136 9363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:57.633933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:34543 (local address 127.0.0.1:60486)
0809 02:02:57.634363 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:57.634379 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 02:02:57.634516 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:02:57.634920 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:02:57.634923 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:02:57.635180 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:02:57.635315 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:57.635322 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:57.636435 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:57.636439 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:57.637395 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:57.637407 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:57.638037 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:57.638041 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:57.638153 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:57.638751 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:02:57.638766 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:02:57.639254 (+ 488us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:02:57.641318 (+ 2064us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:57.641322 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:57.641324 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:57.641629 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:57.641941 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:57.641944 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:57.641946 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:57.642046 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:57.642511 (+ 465us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:02:57.642514 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:02:57.642625 (+ 111us) client_negotiation.cc:769] Sending connection context
0809 02:02:57.642734 (+ 109us) client_negotiation.cc:240] Negotiation successful
0809 02:02:57.642895 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":114,"threads_started":1}
I20260809 02:02:57.643905 8552 tablet_copy_service.cc:140] P 14381b53c95b4bda8c4273431f4a1762: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer d909896773e04b93bbb1e1e7d4c11264 ({username='oryx', principal='oryx/127.1.192.252@KRBTEST.COM'} at 127.0.0.1:60486)
I20260809 02:02:57.644011 8552 tablet_copy_service.cc:161] P 14381b53c95b4bda8c4273431f4a1762: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer d909896773e04b93bbb1e1e7d4c11264 at {username='oryx', principal='oryx/127.1.192.252@KRBTEST.COM'} at 127.0.0.1:60486: session id = d909896773e04b93bbb1e1e7d4c11264-00000000000000000000000000000000
I20260809 02:02:57.645136 8552 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 14381b53c95b4bda8c4273431f4a1762: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 02:02:57.646579 9159 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260809 02:02:57.649161 9159 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P d909896773e04b93bbb1e1e7d4c11264: tablet copy: Starting download of 0 data blocks...
I20260809 02:02:57.649369 9159 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P d909896773e04b93bbb1e1e7d4c11264: tablet copy: Starting download of 1 WAL segments...
I20260809 02:02:57.651494 9159 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P d909896773e04b93bbb1e1e7d4c11264: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 02:02:57.653383 8552 tablet_copy_service.cc:342] P 14381b53c95b4bda8c4273431f4a1762: Request end of tablet copy session d909896773e04b93bbb1e1e7d4c11264-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.1.192.252@KRBTEST.COM'} at 127.0.0.1:60486
I20260809 02:02:57.653509 8552 tablet_copy_service.cc:433] P 14381b53c95b4bda8c4273431f4a1762: ending tablet copy session d909896773e04b93bbb1e1e7d4c11264-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer d909896773e04b93bbb1e1e7d4c11264
I20260809 02:02:57.661221 9159 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:02:57.663259 9364 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:02:57.663400 9365 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:02:57.666750 9370 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:02:57.666945 9159 server_base.cc:1034] running on GCE node
I20260809 02:02:57.667188 9159 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:02:57.667483 9159 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:02:57.668620 9159 hybrid_clock.cc:648] HybridClock initialized: now 1786240977668595 us; error 42 us; skew 500 ppm
Aug 09 02:02:57 dist-test-slave-ss4n krb5kdc[8478](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240977, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.192.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:02:57.670318 9159 init.cc:377] Logged in from keytab as oryx/127.1.192.252@KRBTEST.COM (short username oryx)
I20260809 02:02:57.670799 9159 webserver.cc:460] Webserver started at http://127.1.192.252:39195/ using document root <none> and password file <none>
I20260809 02:02:57.671037 9159 fs_manager.cc:359] Metadata directory not provided
I20260809 02:02:57.671088 9159 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:02:57.672588 9159 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 02:02:57.673686 9378 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:57.673933 9159 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 02:02:57.674017 9159 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/master-2/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/master-2/wal
uuid: "d909896773e04b93bbb1e1e7d4c11264"
format_stamp: "Formatted at 2026-08-09 02:02:57 on dist-test-slave-ss4n"
server_key: "980b5f4100ccdbb6451d0aac9e375e6c"
server_key_iv: "bcd2375561df2b7188e27e61eb1038b6"
server_key_version: "encryptionkey@0"
I20260809 02:02:57.674098 9159 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:02:57.690861 8883 heartbeater.cc:503] Master 127.1.192.254:34543 was elected leader, sending a full tablet report...
I20260809 02:02:57.701095 9159 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:02:57.703631 9159 kserver.cc:163] Server-wide thread pool size limit: 3276
W20260809 02:02:57.708442 9352 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:02:57.708760 9352 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:02:57.708833 9352 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260809 02:02:57.709031 9159 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.252:44581
I20260809 02:02:57.709064 9441 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.252:44581 every 8 connection(s)
I20260809 02:02:57.709689 9159 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/master-2/data/info.pb
I20260809 02:02:57.711491 9442 sys_catalog.cc:263] Verifying existing consensus state
I20260809 02:02:57.712592 9442 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d909896773e04b93bbb1e1e7d4c11264: Bootstrap starting.
W20260809 02:02:57.714936 9352 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:02:57.715013 9352 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:02:57.715108 9352 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.192.193
I20260809 02:02:57.722703 9352 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:44161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/krb5kdc/oryx_127.1.192.193.keytab
--principal=oryx/127.1.192.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.192.193:37543
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.192.193
--webserver_port=35199
--tserver_master_addrs=127.1.192.254:34543,127.1.192.253:40841,127.1.192.252:44581
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9352
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.192.193
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:02:57.724153 9442 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d909896773e04b93bbb1e1e7d4c11264: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 02:02:57.724381 9352 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:02:57.725342 9442 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d909896773e04b93bbb1e1e7d4c11264: Bootstrap complete.
I20260809 02:02:57.725720 9352 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 02:02:57.726109 9442 raft_consensus.cc:348] T 00000000000000000000000000000000 P d909896773e04b93bbb1e1e7d4c11264 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "14381b53c95b4bda8c4273431f4a1762" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 34543 } } peers { permanent_uuid: "bf6ddc1ba97745d897049f1300717b5b" member_type: VOTER last_known_addr { host: "127.1.192.253" port: 40841 } attrs { promote: false } } peers { permanent_uuid: "d909896773e04b93bbb1e1e7d4c11264" member_type: NON_VOTER last_known_addr { host: "127.1.192.252" port: 44581 } attrs { promote: true } }
I20260809 02:02:57.726267 9442 raft_consensus.cc:729] T 00000000000000000000000000000000 P d909896773e04b93bbb1e1e7d4c11264 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: d909896773e04b93bbb1e1e7d4c11264, State: Initialized, Role: LEARNER
I20260809 02:02:57.726468 9442 consensus_queue.cc:260] T 00000000000000000000000000000000 P d909896773e04b93bbb1e1e7d4c11264 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "14381b53c95b4bda8c4273431f4a1762" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 34543 } } peers { permanent_uuid: "bf6ddc1ba97745d897049f1300717b5b" member_type: VOTER last_known_addr { host: "127.1.192.253" port: 40841 } attrs { promote: false } } peers { permanent_uuid: "d909896773e04b93bbb1e1e7d4c11264" member_type: NON_VOTER last_known_addr { host: "127.1.192.252" port: 44581 } attrs { promote: true } }
I20260809 02:02:57.727029 9442 sys_catalog.cc:564] T 00000000000000000000000000000000 P d909896773e04b93bbb1e1e7d4c11264 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:02:57.726984 9445 sys_catalog.cc:455] T 00000000000000000000000000000000 P d909896773e04b93bbb1e1e7d4c11264 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "14381b53c95b4bda8c4273431f4a1762" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 34543 } } peers { permanent_uuid: "bf6ddc1ba97745d897049f1300717b5b" member_type: VOTER last_known_addr { host: "127.1.192.253" port: 40841 } attrs { promote: false } } peers { permanent_uuid: "d909896773e04b93bbb1e1e7d4c11264" member_type: NON_VOTER last_known_addr { host: "127.1.192.252" port: 44581 } attrs { promote: true } } }
I20260809 02:02:57.727174 9445 sys_catalog.cc:458] T 00000000000000000000000000000000 P d909896773e04b93bbb1e1e7d4c11264 [sys.catalog]: This master's current role is: LEARNER
I20260809 02:02:57.730014 9456 catalog_manager.cc:1259] Loaded cluster ID: 279e275c39804b08b8c01de18b06bcb9
I20260809 02:02:57.730525 9456 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d909896773e04b93bbb1e1e7d4c11264: loading cluster ID for follower catalog manager: success
I20260809 02:02:57.732661 9456 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d909896773e04b93bbb1e1e7d4c11264: acquiring CA information for follower catalog manager: success
I20260809 02:02:57.734109 9456 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d909896773e04b93bbb1e1e7d4c11264: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260809 02:02:57.737406 9461 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:02:57.737511 9458 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:02:57.737449 9459 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:02:57.737972 9352 server_base.cc:1034] running on GCE node
I20260809 02:02:57.738320 9352 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:02:57.739007 9352 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:02:57.740190 9352 hybrid_clock.cc:648] HybridClock initialized: now 1786240977740172 us; error 35 us; skew 500 ppm
Aug 09 02:02:57 dist-test-slave-ss4n krb5kdc[8478](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240977, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.192.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:02:57.743268 9352 init.cc:377] Logged in from keytab as oryx/127.1.192.193@KRBTEST.COM (short username oryx)
I20260809 02:02:57.744138 9352 webserver.cc:460] Webserver started at http://127.1.192.193:35199/ using document root <none> and password file <none>
I20260809 02:02:57.744669 9352 fs_manager.cc:359] Metadata directory not provided
I20260809 02:02:57.744745 9352 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:02:57.748759 9352 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260809 02:02:57.751408 9468 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:57.752853 9352 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 02:02:57.752996 9352 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/ts-0/wal
uuid: "2eb7b1762aec408499a7ed56b7923dfc"
format_stamp: "Formatted at 2026-08-09 02:02:56 on dist-test-slave-ss4n"
server_key: "6b107510f0ea141f7228164128d47383"
server_key_iv: "009f86906502d700fcdceed9851903a0"
server_key_version: "encryptionkey@0"
I20260809 02:02:57.753664 9352 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:02:57.779037 9352 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:02:57.782796 9352 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:02:57.783018 9352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:02:57.783663 9352 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:02:57.784556 9352 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:02:57.784606 9352 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:57.784677 9352 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:02:57.784708 9352 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:57.795481 9352 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.193:37543
I20260809 02:02:57.795499 9600 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.193:37543 every 8 connection(s)
I20260809 02:02:57.796757 9352 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/ts-0/data/info.pb
I20260809 02:02:57.802445 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 9352
I20260809 02:02:57.802634 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 8745
Aug 09 02:02:57 dist-test-slave-ss4n krb5kdc[8478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240977, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.192.193@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM
Aug 09 02:02:57 dist-test-slave-ss4n krb5kdc[8478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240977, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.192.193@KRBTEST.COM for oryx/127.1.192.253@KRBTEST.COM
Aug 09 02:02:57 dist-test-slave-ss4n krb5kdc[8478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240977, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.192.193@KRBTEST.COM for oryx/127.1.192.252@KRBTEST.COM
I20260809 02:02:57.812879 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.192.194:35937
--local_ip_for_outbound_sockets=127.1.192.194
--tserver_master_addrs=127.1.192.254:34543
--webserver_port=42957
--webserver_interface=127.1.192.194
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/krb5kdc/oryx_127.1.192.194.keytab
--principal=oryx/127.1.192.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:44161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.1.192.254:34543,127.1.192.253:40841
--tserver_master_addrs=127.1.192.254:34543,127.1.192.253:40841,127.1.192.252:44581 with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 02:02:57.815910 9330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:57.800510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.193:35289 (local address 127.1.192.253:40841)
0809 02:02:57.800647 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:57.800651 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:57.800771 (+ 120us) server_negotiation.cc:407] Connection header received
0809 02:02:57.801536 (+ 765us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:57.801540 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:57.801601 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:57.801702 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:57.803225 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:57.804131 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:57.806298 (+ 2167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:57.806776 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:57.806883 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:57.811266 (+ 4383us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:02:57.811270 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:02:57.811273 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:02:57.811304 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:02:57.813168 (+ 1864us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:57.813877 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:57.813880 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:57.813882 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:57.813980 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:57.814417 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:57.814421 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:57.814423 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:57.814666 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:02:57.814752 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:57.815604 (+ 852us) server_negotiation.cc:299] Negotiation successful
0809 02:02:57.815740 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 02:02:57.815912 9605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:57.798723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.193:54145 (local address 127.1.192.252:44581)
0809 02:02:57.798986 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:57.798991 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:57.799874 (+ 883us) server_negotiation.cc:407] Connection header received
0809 02:02:57.801179 (+ 1305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:57.801182 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:57.801265 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:57.801362 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:57.805954 (+ 4592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:57.807019 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:57.808059 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:57.808527 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:57.808651 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:57.811651 (+ 3000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:02:57.811655 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:02:57.811657 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:02:57.811689 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:02:57.813620 (+ 1931us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:57.814112 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:57.814115 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:57.814117 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:57.814200 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:57.814514 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:57.814517 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:57.814519 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:57.814740 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:02:57.814809 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:57.815604 (+ 795us) server_negotiation.cc:299] Negotiation successful
0809 02:02:57.815742 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":195,"thread_start_us":89,"threads_started":1}
I20260809 02:02:57.817756 9608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:57.800364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.253:40841 (local address 127.1.192.193:35289)
0809 02:02:57.800674 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:57.800687 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:02:57.800838 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:02:57.801737 (+ 899us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:02:57.801741 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:02:57.802101 (+ 360us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:02:57.803008 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:57.803030 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:57.804337 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:57.804344 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:57.806126 (+ 1782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:57.806142 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:57.807115 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:57.807123 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:57.807442 (+ 319us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:57.808503 (+ 1061us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:02:57.808520 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:02:57.811075 (+ 2555us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:02:57.813323 (+ 2248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:57.813332 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:57.813352 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:57.813758 (+ 406us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:57.814102 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:57.814105 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:57.814107 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:57.814310 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:57.814828 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:02:57.814838 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:02:57.815263 (+ 425us) client_negotiation.cc:769] Sending connection context
0809 02:02:57.816184 (+ 921us) client_negotiation.cc:240] Negotiation successful
0809 02:02:57.816542 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"mutex_wait_us":221,"spinlock_wait_cycles":29568,"thread_start_us":124,"threads_started":1}
I20260809 02:02:57.817768 9606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:57.799187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.252:44581 (local address 127.1.192.193:54145)
0809 02:02:57.799672 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:57.799724 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 02:02:57.800829 (+ 1105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:02:57.801430 (+ 601us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:02:57.801439 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:02:57.802132 (+ 693us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:02:57.805789 (+ 3657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:57.805797 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:57.807155 (+ 1358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:57.807157 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:57.807921 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:57.807930 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:57.808710 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:57.808714 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:57.808813 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:57.809549 (+ 736us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:02:57.809564 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:02:57.811509 (+ 1945us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:02:57.813758 (+ 2249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:57.813762 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:57.813764 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:57.813995 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:57.814317 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:57.814320 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:57.814321 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:57.814408 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:57.814873 (+ 465us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:02:57.814876 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:02:57.815263 (+ 387us) client_negotiation.cc:769] Sending connection context
0809 02:02:57.816084 (+ 821us) client_negotiation.cc:240] Negotiation successful
0809 02:02:57.816593 (+ 509us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"mutex_wait_us":3336,"spinlock_wait_cycles":23168,"thread_start_us":139,"threads_started":1}
I20260809 02:02:57.818894 8572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:57.798721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.193:33623 (local address 127.1.192.254:34543)
0809 02:02:57.798866 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:57.798871 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:57.800340 (+ 1469us) server_negotiation.cc:407] Connection header received
0809 02:02:57.801064 (+ 724us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:57.801067 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:57.801135 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:57.801216 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:57.804369 (+ 3153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:57.805240 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:57.806500 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:57.806848 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:57.806935 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:57.810833 (+ 3898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:02:57.810840 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:02:57.810848 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:02:57.810890 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:02:57.814437 (+ 3547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:57.815078 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:57.815084 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:57.815087 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:57.815199 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:57.815559 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:57.815564 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:57.815568 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:57.815835 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:02:57.815939 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:57.818438 (+ 2499us) server_negotiation.cc:299] Negotiation successful
0809 02:02:57.818607 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 02:02:57.819337 9601 heartbeater.cc:347] Connected to a master server at 127.1.192.252:44581
I20260809 02:02:57.819612 9601 heartbeater.cc:464] Registering TS with master...
I20260809 02:02:57.819793 9607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:57.799186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:34543 (local address 127.1.192.193:33623)
0809 02:02:57.800249 (+ 1063us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:57.800262 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:02:57.800829 (+ 567us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:02:57.801432 (+ 603us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:02:57.801439 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:02:57.802065 (+ 626us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:02:57.804210 (+ 2145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:57.804225 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:57.805382 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:57.805388 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:57.806364 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:57.806373 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:57.807114 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:57.807119 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:57.807430 (+ 311us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:57.808413 (+ 983us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:02:57.808457 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:02:57.810556 (+ 2099us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:02:57.814591 (+ 4035us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:57.814598 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:57.814604 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:57.814923 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:57.815324 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:57.815329 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:57.815333 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:57.815438 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:57.816484 (+ 1046us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:02:57.816487 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:02:57.816614 (+ 127us) client_negotiation.cc:769] Sending connection context
0809 02:02:57.816725 (+ 111us) client_negotiation.cc:240] Negotiation successful
0809 02:02:57.816909 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":979,"mutex_wait_us":1652,"spinlock_wait_cycles":6912,"thread_start_us":788,"threads_started":1}
I20260809 02:02:57.820274 9601 heartbeater.cc:511] Master 127.1.192.252:44581 requested a full tablet report, sending...
I20260809 02:02:57.819337 9603 heartbeater.cc:347] Connected to a master server at 127.1.192.253:40841
I20260809 02:02:57.820413 9603 heartbeater.cc:464] Registering TS with master...
I20260809 02:02:57.820567 9603 heartbeater.cc:511] Master 127.1.192.253:40841 requested a full tablet report, sending...
I20260809 02:02:57.821885 9195 ts_manager.cc:194] Registered new tserver with Master: 2eb7b1762aec408499a7ed56b7923dfc (127.1.192.193:37543)
I20260809 02:02:57.822016 9399 ts_manager.cc:194] Registered new tserver with Master: 2eb7b1762aec408499a7ed56b7923dfc (127.1.192.193:37543)
I20260809 02:02:57.822417 9602 heartbeater.cc:347] Connected to a master server at 127.1.192.254:34543
I20260809 02:02:57.822544 9602 heartbeater.cc:464] Registering TS with master...
I20260809 02:02:57.822724 9602 heartbeater.cc:511] Master 127.1.192.254:34543 requested a full tablet report, sending...
I20260809 02:02:57.823513 8527 ts_manager.cc:194] Re-registered known tserver with Master: 2eb7b1762aec408499a7ed56b7923dfc (127.1.192.193:37543)
I20260809 02:02:57.824070 8527 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.192.193@KRBTEST.COM'} at 127.1.192.193:33623
I20260809 02:02:57.929787 9036 heartbeater.cc:503] Master 127.1.192.254:34543 was elected leader, sending a full tablet report...
W20260809 02:02:57.936749 9609 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:02:57.937036 9609 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:02:57.937122 9609 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:02:57.941264 9609 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:02:57.941347 9609 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:02:57.941432 9609 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.192.194
I20260809 02:02:57.946907 9609 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:44161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/krb5kdc/oryx_127.1.192.194.keytab
--principal=oryx/127.1.192.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.192.194:35937
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.192.194
--webserver_port=42957
--tserver_master_addrs=127.1.192.254:34543,127.1.192.253:40841,127.1.192.252:44581
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9609
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.192.194
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:02:57.948081 9609 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:02:57.949199 9609 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:02:57.959373 9615 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:02:57.959403 9614 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:02:57.959386 9617 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:02:57.959720 9609 server_base.cc:1034] running on GCE node
I20260809 02:02:57.960239 9609 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:02:57.960916 9609 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:02:57.962126 9609 hybrid_clock.cc:648] HybridClock initialized: now 1786240977962099 us; error 39 us; skew 500 ppm
Aug 09 02:02:57 dist-test-slave-ss4n krb5kdc[8478](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240977, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.192.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:02:57.965359 9609 init.cc:377] Logged in from keytab as oryx/127.1.192.194@KRBTEST.COM (short username oryx)
I20260809 02:02:57.966544 9609 webserver.cc:460] Webserver started at http://127.1.192.194:42957/ using document root <none> and password file <none>
I20260809 02:02:57.967193 9609 fs_manager.cc:359] Metadata directory not provided
I20260809 02:02:57.967332 9609 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:02:57.971972 9609 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260809 02:02:57.974618 9625 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:57.976007 9609 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 02:02:57.976162 9609 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/ts-1/wal
uuid: "8030853bc1144362a198609d5c2a1a6d"
format_stamp: "Formatted at 2026-08-09 02:02:56 on dist-test-slave-ss4n"
server_key: "632d84def7d21001f400bf731ee909e1"
server_key_iv: "188ef141ac4dd8c74af4dbcb7f330f48"
server_key_version: "encryptionkey@0"
I20260809 02:02:57.976748 9609 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:02:57.995842 9609 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:02:57.999691 9609 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:02:57.999922 9609 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:02:58.000563 9609 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:02:58.001456 9609 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:02:58.001747 9609 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:58.001880 9609 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:02:58.001957 9609 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:58.013266 9609 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.194:35937
I20260809 02:02:58.013307 9746 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.194:35937 every 8 connection(s)
I20260809 02:02:58.014776 9609 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/ts-1/data/info.pb
I20260809 02:02:58.022684 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 9609
I20260809 02:02:58.022868 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 8889
I20260809 02:02:58.035122 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.192.195:39329
--local_ip_for_outbound_sockets=127.1.192.195
--tserver_master_addrs=127.1.192.254:34543
--webserver_port=45817
--webserver_interface=127.1.192.195
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/krb5kdc/oryx_127.1.192.195.keytab
--principal=oryx/127.1.192.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:44161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.1.192.254:34543,127.1.192.253:40841
--tserver_master_addrs=127.1.192.254:34543,127.1.192.253:40841,127.1.192.252:44581 with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 02:02:58 dist-test-slave-ss4n krb5kdc[8478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240977, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.192.194@KRBTEST.COM for oryx/127.1.192.252@KRBTEST.COM
Aug 09 02:02:58 dist-test-slave-ss4n krb5kdc[8478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240977, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.192.194@KRBTEST.COM for oryx/127.1.192.253@KRBTEST.COM
Aug 09 02:02:58 dist-test-slave-ss4n krb5kdc[8478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240977, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.192.194@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM
I20260809 02:02:58.045008 9330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:58.018543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.194:35147 (local address 127.1.192.253:40841)
0809 02:02:58.018682 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:58.018687 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:58.018895 (+ 208us) server_negotiation.cc:407] Connection header received
0809 02:02:58.019519 (+ 624us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:58.019527 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:58.019656 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:58.019791 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:58.021307 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:58.021929 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:58.034746 (+ 12817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:58.035092 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:58.035220 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:58.039855 (+ 4635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:02:58.039859 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:02:58.039862 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:02:58.039892 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:02:58.042250 (+ 2358us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:58.042999 (+ 749us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:58.043003 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:58.043005 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:58.043101 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:58.043562 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:58.043565 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:58.043567 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:58.043799 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:02:58.043876 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:58.044692 (+ 816us) server_negotiation.cc:299] Negotiation successful
0809 02:02:58.044830 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 02:02:58.045291 9605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:58.016911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.194:44621 (local address 127.1.192.252:44581)
0809 02:02:58.017074 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:58.017080 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:58.018178 (+ 1098us) server_negotiation.cc:407] Connection header received
0809 02:02:58.019470 (+ 1292us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:58.019474 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:58.019528 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:58.019588 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:58.021344 (+ 1756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:58.022290 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:58.035007 (+ 12717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:58.035406 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:58.035550 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:58.039334 (+ 3784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:02:58.039341 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:02:58.039347 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:02:58.039385 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:02:58.042003 (+ 2618us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:58.042984 (+ 981us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:58.042990 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:58.042994 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:58.043098 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:58.043665 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:58.043671 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:58.043675 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:58.043949 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:02:58.044056 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:58.044882 (+ 826us) server_negotiation.cc:299] Negotiation successful
0809 02:02:58.045054 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 02:02:58.046284 8572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:58.016824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.194:50851 (local address 127.1.192.254:34543)
0809 02:02:58.016993 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:58.016999 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:58.018475 (+ 1476us) server_negotiation.cc:407] Connection header received
0809 02:02:58.019459 (+ 984us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:58.019464 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:58.019528 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:58.019588 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:58.021343 (+ 1755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:58.022303 (+ 960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:58.036408 (+ 14105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:58.036821 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:58.036928 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:58.039855 (+ 2927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:02:58.039859 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:02:58.039862 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:02:58.039892 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:02:58.042159 (+ 2267us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:58.043000 (+ 841us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:58.043003 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:58.043005 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:58.043101 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:58.043576 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:58.043579 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:58.043581 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:58.043799 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:02:58.043882 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:58.045967 (+ 2085us) server_negotiation.cc:299] Negotiation successful
0809 02:02:58.046140 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 02:02:58.047297 9753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:58.017263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:34543 (local address 127.1.192.194:50851)
0809 02:02:58.018377 (+ 1114us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:58.018391 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 02:02:58.019203 (+ 812us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:02:58.019939 (+ 736us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:02:58.019949 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:02:58.020576 (+ 627us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:02:58.021168 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:58.021175 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:58.030252 (+ 9077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:58.030256 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:58.036228 (+ 5972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:58.036240 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:58.037137 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:58.037141 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:58.037269 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:58.037945 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:02:58.037960 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:02:58.039668 (+ 1708us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:02:58.042318 (+ 2650us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:58.042322 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:58.042324 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:58.042868 (+ 544us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:58.043219 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:58.043222 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:58.043225 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:58.043443 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:58.043957 (+ 514us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:02:58.043964 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:02:58.044368 (+ 404us) client_negotiation.cc:769] Sending connection context
0809 02:02:58.046510 (+ 2142us) client_negotiation.cc:240] Negotiation successful
0809 02:02:58.046676 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1032,"mutex_wait_us":3981,"spinlock_wait_cycles":28800,"thread_start_us":234,"threads_started":1}
I20260809 02:02:58.047298 9751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:58.017263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.252:44581 (local address 127.1.192.194:44621)
0809 02:02:58.017979 (+ 716us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:58.018051 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0809 02:02:58.019203 (+ 1152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:02:58.019939 (+ 736us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:02:58.019949 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:02:58.020576 (+ 627us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:02:58.021165 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:58.021175 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:58.022476 (+ 1301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:58.022480 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:58.034841 (+ 12361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:58.034854 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:58.035756 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:58.035763 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:58.035904 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:58.036455 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:02:58.036470 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:02:58.039106 (+ 2636us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:02:58.042186 (+ 3080us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:58.042197 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:58.042220 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:58.042835 (+ 615us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:58.043234 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:58.043238 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:58.043242 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:58.043532 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:58.044128 (+ 596us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:02:58.044132 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:02:58.044384 (+ 252us) client_negotiation.cc:769] Sending connection context
0809 02:02:58.045472 (+ 1088us) client_negotiation.cc:240] Negotiation successful
0809 02:02:58.046046 (+ 574us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":453,"mutex_wait_us":115,"spinlock_wait_cycles":158720,"thread_start_us":768,"threads_started":1}
I20260809 02:02:58.048343 9754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:58.018452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.253:40841 (local address 127.1.192.194:35147)
0809 02:02:58.018787 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:58.018801 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 02:02:58.019220 (+ 419us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:02:58.019964 (+ 744us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:02:58.019967 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:02:58.020581 (+ 614us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:02:58.021155 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:58.021174 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:58.022072 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:58.022075 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:58.034521 (+ 12446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:58.034542 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:58.035310 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:58.035316 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:58.035584 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:58.036321 (+ 737us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:02:58.036350 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:02:58.039668 (+ 3318us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:02:58.042362 (+ 2694us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:58.042367 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:58.042372 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:58.042868 (+ 496us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:58.043219 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:58.043222 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:58.043225 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:58.043443 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:58.043957 (+ 514us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:02:58.043971 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:02:58.044383 (+ 412us) client_negotiation.cc:769] Sending connection context
0809 02:02:58.045766 (+ 1383us) client_negotiation.cc:240] Negotiation successful
0809 02:02:58.046142 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"mutex_wait_us":11360,"spinlock_wait_cycles":61184,"thread_start_us":120,"threads_started":1}
I20260809 02:02:58.048746 9747 heartbeater.cc:347] Connected to a master server at 127.1.192.252:44581
I20260809 02:02:58.048933 9748 heartbeater.cc:347] Connected to a master server at 127.1.192.254:34543
I20260809 02:02:58.049013 9747 heartbeater.cc:464] Registering TS with master...
I20260809 02:02:58.049021 9748 heartbeater.cc:464] Registering TS with master...
I20260809 02:02:58.049319 9749 heartbeater.cc:347] Connected to a master server at 127.1.192.253:40841
I20260809 02:02:58.049391 9749 heartbeater.cc:464] Registering TS with master...
I20260809 02:02:58.049679 9747 heartbeater.cc:511] Master 127.1.192.252:44581 requested a full tablet report, sending...
I20260809 02:02:58.050300 9748 heartbeater.cc:511] Master 127.1.192.254:34543 requested a full tablet report, sending...
I20260809 02:02:58.050477 9749 heartbeater.cc:511] Master 127.1.192.253:40841 requested a full tablet report, sending...
I20260809 02:02:58.051034 8527 ts_manager.cc:194] Re-registered known tserver with Master: 8030853bc1144362a198609d5c2a1a6d (127.1.192.194:35937)
I20260809 02:02:58.051064 9399 ts_manager.cc:194] Registered new tserver with Master: 8030853bc1144362a198609d5c2a1a6d (127.1.192.194:35937)
I20260809 02:02:58.051427 9195 ts_manager.cc:194] Registered new tserver with Master: 8030853bc1144362a198609d5c2a1a6d (127.1.192.194:35937)
I20260809 02:02:58.051834 8527 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.192.194@KRBTEST.COM'} at 127.1.192.194:50851
I20260809 02:02:58.125169 9756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:58.121281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.252:44581 (local address 127.0.0.1:32856)
0809 02:02:58.121674 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:58.121690 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 02:02:58.121844 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:02:58.122139 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:02:58.122142 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:02:58.122162 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:02:58.122291 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:58.122296 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:58.123045 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:58.123049 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:58.124033 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:58.124044 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:58.124759 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:58.124762 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:58.124851 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:58.124876 (+ 25us) client_negotiation.cc:769] Sending connection context
0809 02:02:58.124941 (+ 65us) client_negotiation.cc:240] Negotiation successful
0809 02:02:58.125038 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":102,"threads_started":1}
I20260809 02:02:58.125166 9605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:58.121443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32856 (local address 127.1.192.252:44581)
0809 02:02:58.121674 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:58.121681 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:58.121778 (+ 97us) server_negotiation.cc:407] Connection header received
0809 02:02:58.121954 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:58.121962 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:58.122015 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:58.122113 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:02:58.122402 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:58.122929 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:58.124160 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:58.124624 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:58.124753 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:58.124917 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:58.124957 (+ 40us) server_negotiation.cc:299] Negotiation successful
0809 02:02:58.125024 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146}
I20260809 02:02:58.125847 9412 raft_consensus.cc:1203] T 00000000000000000000000000000000 P d909896773e04b93bbb1e1e7d4c11264 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260809 02:02:58.126224 9445 sys_catalog.cc:455] T 00000000000000000000000000000000 P d909896773e04b93bbb1e1e7d4c11264 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 14381b53c95b4bda8c4273431f4a1762. Latest consensus state: current_term: 1 leader_uuid: "14381b53c95b4bda8c4273431f4a1762" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "14381b53c95b4bda8c4273431f4a1762" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 34543 } } peers { permanent_uuid: "bf6ddc1ba97745d897049f1300717b5b" member_type: VOTER last_known_addr { host: "127.1.192.253" port: 40841 } attrs { promote: false } } peers { permanent_uuid: "d909896773e04b93bbb1e1e7d4c11264" member_type: NON_VOTER last_known_addr { host: "127.1.192.252" port: 44581 } attrs { promote: true } } }
I20260809 02:02:58.126387 9445 sys_catalog.cc:458] T 00000000000000000000000000000000 P d909896773e04b93bbb1e1e7d4c11264 [sys.catalog]: This master's current role is: LEARNER
W20260809 02:02:58.154765 9755 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:02:58.155103 9755 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:02:58.155192 9755 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:02:58.159408 9755 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:02:58.159507 9755 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:02:58.159603 9755 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.192.195
I20260809 02:02:58.165321 9755 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:44161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/krb5kdc/oryx_127.1.192.195.keytab
--principal=oryx/127.1.192.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.192.195:39329
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.192.195
--webserver_port=45817
--tserver_master_addrs=127.1.192.254:34543,127.1.192.253:40841,127.1.192.252:44581
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9755
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.192.195
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:02:58.166486 9755 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:02:58.167603 9755 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:02:58.177568 9761 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:02:58.177639 9762 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:02:58.177646 9764 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:02:58.178054 9755 server_base.cc:1034] running on GCE node
I20260809 02:02:58.178473 9755 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:02:58.179158 9755 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:02:58.180353 9755 hybrid_clock.cc:648] HybridClock initialized: now 1786240978180329 us; error 36 us; skew 500 ppm
Aug 09 02:02:58 dist-test-slave-ss4n krb5kdc[8478](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240978, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.192.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:02:58.183645 9755 init.cc:377] Logged in from keytab as oryx/127.1.192.195@KRBTEST.COM (short username oryx)
I20260809 02:02:58.184525 9755 webserver.cc:460] Webserver started at http://127.1.192.195:45817/ using document root <none> and password file <none>
I20260809 02:02:58.185034 9755 fs_manager.cc:359] Metadata directory not provided
I20260809 02:02:58.185106 9755 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:02:58.188977 9755 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260809 02:02:58.191116 9772 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:58.192480 9755 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 02:02:58.192587 9755 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/ts-2/wal
uuid: "cd2035d243e94eeea383bb2f79f51dc1"
format_stamp: "Formatted at 2026-08-09 02:02:56 on dist-test-slave-ss4n"
server_key: "96da89dd767bab8c7ca9c8146f1b2cc5"
server_key_iv: "3f11469d24ed1d6f316e96710f3a7854"
server_key_version: "encryptionkey@0"
I20260809 02:02:58.193231 9755 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:02:58.210350 9755 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:02:58.213891 9755 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:02:58.214107 9755 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:02:58.214800 9755 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:02:58.215709 9755 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:02:58.215759 9755 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:58.215798 9755 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:02:58.215857 9755 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:02:58.225946 9755 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.195:39329
I20260809 02:02:58.225971 9900 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.195:39329 every 8 connection(s)
I20260809 02:02:58.227056 9755 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/ts-2/data/info.pb
I20260809 02:02:58.233680 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 9755
Aug 09 02:02:58 dist-test-slave-ss4n krb5kdc[8478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240978, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.192.195@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM
Aug 09 02:02:58 dist-test-slave-ss4n krb5kdc[8478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240978, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.192.195@KRBTEST.COM for oryx/127.1.192.252@KRBTEST.COM
Aug 09 02:02:58 dist-test-slave-ss4n krb5kdc[8478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240978, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.192.195@KRBTEST.COM for oryx/127.1.192.253@KRBTEST.COM
I20260809 02:02:58.245555 8572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:58.230902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.195:37711 (local address 127.1.192.254:34543)
0809 02:02:58.231064 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:58.231069 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:58.231106 (+ 37us) server_negotiation.cc:407] Connection header received
0809 02:02:58.231574 (+ 468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:58.231578 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:58.231653 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:58.231707 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:58.233067 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:58.233908 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:58.235803 (+ 1895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:58.236129 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:58.236203 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:58.240870 (+ 4667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:02:58.240876 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:02:58.240881 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:02:58.240914 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:02:58.242728 (+ 1814us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:58.243620 (+ 892us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:58.243627 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:58.243629 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:58.243724 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:58.244150 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:58.244153 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:58.244155 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:58.244386 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:02:58.244459 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:58.245227 (+ 768us) server_negotiation.cc:299] Negotiation successful
0809 02:02:58.245350 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 02:02:58.245555 9605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:58.229306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.195:34785 (local address 127.1.192.252:44581)
0809 02:02:58.229458 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:58.229463 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:58.230368 (+ 905us) server_negotiation.cc:407] Connection header received
0809 02:02:58.231574 (+ 1206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:58.231578 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:58.231637 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:58.231696 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:58.233067 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:58.233908 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:58.236608 (+ 2700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:58.236961 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:58.237037 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:58.240286 (+ 3249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:02:58.240294 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:02:58.240298 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:02:58.240326 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:02:58.243580 (+ 3254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:58.244061 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:58.244067 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:58.244072 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:58.244177 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:58.244489 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:58.244496 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:58.244497 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:58.244703 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:02:58.244773 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:58.245227 (+ 454us) server_negotiation.cc:299] Negotiation successful
0809 02:02:58.245350 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 02:02:58.246165 9330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:58.229471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.195:39509 (local address 127.1.192.253:40841)
0809 02:02:58.229622 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:58.229627 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:58.230364 (+ 737us) server_negotiation.cc:407] Connection header received
0809 02:02:58.231567 (+ 1203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:58.231571 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:58.231631 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:58.231708 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:58.233043 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:58.233748 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:58.235618 (+ 1870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:58.235958 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:58.236050 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:58.240642 (+ 4592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:02:58.240647 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:02:58.240650 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:02:58.240679 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:02:58.242853 (+ 2174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:58.243590 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:58.243594 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:58.243595 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:58.243692 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:58.244323 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:58.244327 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:58.244328 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:58.244569 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:02:58.244670 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:58.245885 (+ 1215us) server_negotiation.cc:299] Negotiation successful
0809 02:02:58.245998 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 02:02:58.246902 9908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:58.230757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:34543 (local address 127.1.192.195:37711)
0809 02:02:58.231030 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:58.231042 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:02:58.231363 (+ 321us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:02:58.231880 (+ 517us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:02:58.231887 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:02:58.232396 (+ 509us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:02:58.232909 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:58.232917 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:58.234060 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:58.234066 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:58.235695 (+ 1629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:58.235701 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:58.236400 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:58.236404 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:58.236644 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:58.237391 (+ 747us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:02:58.237414 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:02:58.240709 (+ 3295us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:02:58.243053 (+ 2344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:58.243057 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:58.243059 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:58.243479 (+ 420us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:58.243839 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:58.243842 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:58.243847 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:58.244039 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:58.244547 (+ 508us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:02:58.244554 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:02:58.244911 (+ 357us) client_negotiation.cc:769] Sending connection context
0809 02:02:58.245717 (+ 806us) client_negotiation.cc:240] Negotiation successful
0809 02:02:58.245856 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"mutex_wait_us":614,"spinlock_wait_cycles":16768,"thread_start_us":87,"threads_started":1}
I20260809 02:02:58.246920 9906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:58.229704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.252:44581 (local address 127.1.192.195:34785)
0809 02:02:58.230204 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:58.230257 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 02:02:58.231363 (+ 1106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:02:58.231879 (+ 516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:02:58.231887 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:02:58.232398 (+ 511us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:02:58.232910 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:58.232917 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:58.234133 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:58.234136 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:58.236480 (+ 2344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:58.236487 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:58.237190 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:58.237192 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:58.237305 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:58.237718 (+ 413us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:02:58.237735 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:02:58.240023 (+ 2288us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:02:58.243709 (+ 3686us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:58.243713 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:58.243715 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:58.243944 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:58.244297 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:58.244299 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:58.244301 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:58.244379 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:58.244848 (+ 469us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:02:58.244851 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:02:58.244925 (+ 74us) client_negotiation.cc:769] Sending connection context
0809 02:02:58.246027 (+ 1102us) client_negotiation.cc:240] Negotiation successful
0809 02:02:58.246170 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"mutex_wait_us":1368,"spinlock_wait_cycles":53376,"thread_start_us":132,"threads_started":1}
I20260809 02:02:58.247489 9907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:58.229699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.253:40841 (local address 127.1.192.195:39509)
0809 02:02:58.230203 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:58.230258 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0809 02:02:58.231345 (+ 1087us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:02:58.231878 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:02:58.231885 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:02:58.232373 (+ 488us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:02:58.232900 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:58.232913 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:58.233967 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:58.233971 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:58.235492 (+ 1521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:58.235501 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:58.236400 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:58.236404 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:58.236643 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:58.237788 (+ 1145us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:02:58.237806 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:02:58.240342 (+ 2536us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:02:58.242971 (+ 2629us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:58.242977 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:58.242994 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:58.243480 (+ 486us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:58.244076 (+ 596us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:02:58.244080 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:02:58.244081 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:02:58.244182 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:02:58.244722 (+ 540us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:02:58.244727 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:02:58.244912 (+ 185us) client_negotiation.cc:769] Sending connection context
0809 02:02:58.245139 (+ 227us) client_negotiation.cc:240] Negotiation successful
0809 02:02:58.245526 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":344,"mutex_wait_us":218,"spinlock_wait_cycles":47872,"thread_start_us":151,"threads_started":1}
I20260809 02:02:58.248524 9902 heartbeater.cc:347] Connected to a master server at 127.1.192.254:34543
I20260809 02:02:58.248838 9902 heartbeater.cc:464] Registering TS with master...
I20260809 02:02:58.249500 9902 heartbeater.cc:511] Master 127.1.192.254:34543 requested a full tablet report, sending...
I20260809 02:02:58.248569 9901 heartbeater.cc:347] Connected to a master server at 127.1.192.252:44581
I20260809 02:02:58.250190 9901 heartbeater.cc:464] Registering TS with master...
I20260809 02:02:58.250377 9901 heartbeater.cc:511] Master 127.1.192.252:44581 requested a full tablet report, sending...
I20260809 02:02:58.248548 9904 heartbeater.cc:347] Connected to a master server at 127.1.192.253:40841
I20260809 02:02:58.250603 9904 heartbeater.cc:464] Registering TS with master...
I20260809 02:02:58.250767 9904 heartbeater.cc:511] Master 127.1.192.253:40841 requested a full tablet report, sending...
I20260809 02:02:58.251747 9399 ts_manager.cc:194] Registered new tserver with Master: cd2035d243e94eeea383bb2f79f51dc1 (127.1.192.195:39329)
I20260809 02:02:58.251813 9195 ts_manager.cc:194] Registered new tserver with Master: cd2035d243e94eeea383bb2f79f51dc1 (127.1.192.195:39329)
I20260809 02:02:58.251937 8527 ts_manager.cc:194] Re-registered known tserver with Master: cd2035d243e94eeea383bb2f79f51dc1 (127.1.192.195:39329)
I20260809 02:02:58.252671 8527 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.192.195@KRBTEST.COM'} at 127.1.192.195:37711
I20260809 02:02:58.253103 9919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:58.236202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48596 (local address 127.1.192.253:40841)
0809 02:02:58.237151 (+ 949us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:58.237156 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:58.239573 (+ 2417us) server_negotiation.cc:407] Connection header received
0809 02:02:58.239623 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:58.239627 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:58.239684 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:58.239768 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:58.240676 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:58.241535 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:58.242853 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:58.243208 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:58.243298 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:58.248315 (+ 5017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:02:58.248321 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:02:58.248325 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:02:58.248358 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:02:58.250737 (+ 2379us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:58.251319 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:58.251325 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:58.251330 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:58.251426 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:58.251824 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:58.251828 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:58.251829 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:58.252104 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:02:58.252214 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:58.252808 (+ 594us) server_negotiation.cc:299] Negotiation successful
0809 02:02:58.252936 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":865,"thread_start_us":89,"threads_started":1}
I20260809 02:02:58.255146 9918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:58.236219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60490 (local address 127.1.192.254:34543)
0809 02:02:58.237296 (+ 1077us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:58.237302 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:58.237318 (+ 16us) server_negotiation.cc:407] Connection header received
0809 02:02:58.237364 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:58.237368 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:58.237416 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:58.237538 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:58.238683 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:58.243918 (+ 5235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:58.246460 (+ 2542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:58.246804 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:58.248081 (+ 1277us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:58.250007 (+ 1926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:02:58.250010 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:02:58.250013 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:02:58.250040 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:02:58.253276 (+ 3236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:58.253789 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:58.253793 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:58.253796 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:58.253899 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:58.254241 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:58.254244 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:58.254245 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:58.254408 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:02:58.254564 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:58.254814 (+ 250us) server_negotiation.cc:299] Negotiation successful
0809 02:02:58.254949 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1002,"thread_start_us":68,"threads_started":1}
I20260809 02:02:58.256675 9914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:58.235712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32866 (local address 127.1.192.252:44581)
0809 02:02:58.236331 (+ 619us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:58.236335 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:58.238868 (+ 2533us) server_negotiation.cc:407] Connection header received
0809 02:02:58.239002 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:58.239005 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:58.239057 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:58.239152 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:58.240112 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:58.241755 (+ 1643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:58.247656 (+ 5901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:58.247983 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:58.248432 (+ 449us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:58.250522 (+ 2090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:02:58.250530 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:02:58.250783 (+ 253us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:02:58.250836 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:02:58.254760 (+ 3924us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:58.255369 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:58.255372 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:58.255373 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:58.255482 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:58.255757 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:58.255760 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:58.255762 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:58.255998 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:02:58.256105 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:58.256339 (+ 234us) server_negotiation.cc:299] Negotiation successful
0809 02:02:58.256483 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":533,"thread_start_us":72,"threads_started":1}
I20260809 02:02:58.258138 8527 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60490:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 02:02:58.261157 8527 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:02:58.264000 9153 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14381b53c95b4bda8c4273431f4a1762 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "14381b53c95b4bda8c4273431f4a1762" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "14381b53c95b4bda8c4273431f4a1762" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 34543 } } peers { permanent_uuid: "bf6ddc1ba97745d897049f1300717b5b" member_type: VOTER last_known_addr { host: "127.1.192.253" port: 40841 } attrs { promote: false } } peers { permanent_uuid: "d909896773e04b93bbb1e1e7d4c11264" member_type: NON_VOTER last_known_addr { host: "127.1.192.252" port: 44581 } attrs { promote: true } } }
I20260809 02:02:58.264150 9153 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14381b53c95b4bda8c4273431f4a1762 [sys.catalog]: This master's current role is: LEADER
I20260809 02:02:58.267612 9926 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 14381b53c95b4bda8c4273431f4a1762: attempting to promote NON_VOTER d909896773e04b93bbb1e1e7d4c11264 to VOTER
I20260809 02:02:58.267956 9926 consensus_queue.cc:237] T 00000000000000000000000000000000 P 14381b53c95b4bda8c4273431f4a1762 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "14381b53c95b4bda8c4273431f4a1762" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 34543 } } peers { permanent_uuid: "bf6ddc1ba97745d897049f1300717b5b" member_type: VOTER last_known_addr { host: "127.1.192.253" port: 40841 } attrs { promote: false } } peers { permanent_uuid: "d909896773e04b93bbb1e1e7d4c11264" member_type: VOTER last_known_addr { host: "127.1.192.252" port: 44581 } attrs { promote: false } }
I20260809 02:02:58.269075 9412 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d909896773e04b93bbb1e1e7d4c11264 [term 1 LEARNER]: Refusing update from remote peer 14381b53c95b4bda8c4273431f4a1762: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260809 02:02:58.269462 9925 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 14381b53c95b4bda8c4273431f4a1762 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d909896773e04b93bbb1e1e7d4c11264" member_type: VOTER last_known_addr { host: "127.1.192.252" port: 44581 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260809 02:02:58.271106 9924 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 14381b53c95b4bda8c4273431f4a1762 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, d909896773e04b93bbb1e1e7d4c11264 (127.1.192.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "14381b53c95b4bda8c4273431f4a1762" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 34543 } } peers { permanent_uuid: "bf6ddc1ba97745d897049f1300717b5b" member_type: VOTER last_known_addr { host: "127.1.192.253" port: 40841 } attrs { promote: false } } peers { permanent_uuid: "d909896773e04b93bbb1e1e7d4c11264" member_type: VOTER last_known_addr { host: "127.1.192.252" port: 44581 } attrs { promote: false } } }
I20260809 02:02:58.271289 9204 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bf6ddc1ba97745d897049f1300717b5b [term 1 FOLLOWER]: Refusing update from remote peer 14381b53c95b4bda8c4273431f4a1762: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260809 02:02:58.271700 9926 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 14381b53c95b4bda8c4273431f4a1762 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf6ddc1ba97745d897049f1300717b5b" member_type: VOTER last_known_addr { host: "127.1.192.253" port: 40841 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260809 02:02:58.272758 9924 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14381b53c95b4bda8c4273431f4a1762 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "14381b53c95b4bda8c4273431f4a1762" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "14381b53c95b4bda8c4273431f4a1762" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 34543 } } peers { permanent_uuid: "bf6ddc1ba97745d897049f1300717b5b" member_type: VOTER last_known_addr { host: "127.1.192.253" port: 40841 } attrs { promote: false } } peers { permanent_uuid: "d909896773e04b93bbb1e1e7d4c11264" member_type: VOTER last_known_addr { host: "127.1.192.252" port: 44581 } attrs { promote: false } } }
I20260809 02:02:58.272883 9924 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14381b53c95b4bda8c4273431f4a1762 [sys.catalog]: This master's current role is: LEADER
I20260809 02:02:58.272778 9204 raft_consensus.cc:2932] T 00000000000000000000000000000000 P bf6ddc1ba97745d897049f1300717b5b [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, d909896773e04b93bbb1e1e7d4c11264 (127.1.192.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "14381b53c95b4bda8c4273431f4a1762" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 34543 } } peers { permanent_uuid: "bf6ddc1ba97745d897049f1300717b5b" member_type: VOTER last_known_addr { host: "127.1.192.253" port: 40841 } attrs { promote: false } } peers { permanent_uuid: "d909896773e04b93bbb1e1e7d4c11264" member_type: VOTER last_known_addr { host: "127.1.192.252" port: 44581 } attrs { promote: false } } }
I20260809 02:02:58.273022 9924 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14381b53c95b4bda8c4273431f4a1762 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "14381b53c95b4bda8c4273431f4a1762" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "14381b53c95b4bda8c4273431f4a1762" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 34543 } } peers { permanent_uuid: "bf6ddc1ba97745d897049f1300717b5b" member_type: VOTER last_known_addr { host: "127.1.192.253" port: 40841 } attrs { promote: false } } peers { permanent_uuid: "d909896773e04b93bbb1e1e7d4c11264" member_type: VOTER last_known_addr { host: "127.1.192.252" port: 44581 } attrs { promote: false } } }
I20260809 02:02:58.273138 9924 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14381b53c95b4bda8c4273431f4a1762 [sys.catalog]: This master's current role is: LEADER
I20260809 02:02:58.273789 9412 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d909896773e04b93bbb1e1e7d4c11264 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, d909896773e04b93bbb1e1e7d4c11264 (127.1.192.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "14381b53c95b4bda8c4273431f4a1762" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 34543 } } peers { permanent_uuid: "bf6ddc1ba97745d897049f1300717b5b" member_type: VOTER last_known_addr { host: "127.1.192.253" port: 40841 } attrs { promote: false } } peers { permanent_uuid: "d909896773e04b93bbb1e1e7d4c11264" member_type: VOTER last_known_addr { host: "127.1.192.252" port: 44581 } attrs { promote: false } } }
I20260809 02:02:58.274381 9934 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf6ddc1ba97745d897049f1300717b5b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "14381b53c95b4bda8c4273431f4a1762" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "14381b53c95b4bda8c4273431f4a1762" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 34543 } } peers { permanent_uuid: "bf6ddc1ba97745d897049f1300717b5b" member_type: VOTER last_known_addr { host: "127.1.192.253" port: 40841 } attrs { promote: false } } peers { permanent_uuid: "d909896773e04b93bbb1e1e7d4c11264" member_type: VOTER last_known_addr { host: "127.1.192.252" port: 44581 } attrs { promote: false } } }
I20260809 02:02:58.274516 9934 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf6ddc1ba97745d897049f1300717b5b [sys.catalog]: This master's current role is: FOLLOWER
I20260809 02:02:58.275357 9924 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14381b53c95b4bda8c4273431f4a1762 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "14381b53c95b4bda8c4273431f4a1762" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "14381b53c95b4bda8c4273431f4a1762" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 34543 } } peers { permanent_uuid: "bf6ddc1ba97745d897049f1300717b5b" member_type: VOTER last_known_addr { host: "127.1.192.253" port: 40841 } attrs { promote: false } } peers { permanent_uuid: "d909896773e04b93bbb1e1e7d4c11264" member_type: VOTER last_known_addr { host: "127.1.192.252" port: 44581 } attrs { promote: false } } }
I20260809 02:02:58.275465 9924 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14381b53c95b4bda8c4273431f4a1762 [sys.catalog]: This master's current role is: LEADER
I20260809 02:02:58.276064 9445 sys_catalog.cc:455] T 00000000000000000000000000000000 P d909896773e04b93bbb1e1e7d4c11264 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "14381b53c95b4bda8c4273431f4a1762" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "14381b53c95b4bda8c4273431f4a1762" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 34543 } } peers { permanent_uuid: "bf6ddc1ba97745d897049f1300717b5b" member_type: VOTER last_known_addr { host: "127.1.192.253" port: 40841 } attrs { promote: false } } peers { permanent_uuid: "d909896773e04b93bbb1e1e7d4c11264" member_type: VOTER last_known_addr { host: "127.1.192.252" port: 44581 } attrs { promote: false } } }
I20260809 02:02:58.276180 9445 sys_catalog.cc:458] T 00000000000000000000000000000000 P d909896773e04b93bbb1e1e7d4c11264 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 02:02:58.283408 9756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:58.278029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.193:37543 (local address 127.0.0.1:41758)
0809 02:02:58.278547 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:58.278567 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 02:02:58.278652 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:02:58.279686 (+ 1034us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:02:58.279689 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:02:58.279707 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:02:58.279839 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:58.279846 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:58.280908 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:58.280914 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:58.281860 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:58.281870 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:58.283022 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:58.283026 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:58.283118 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:58.283144 (+ 26us) client_negotiation.cc:769] Sending connection context
0809 02:02:58.283191 (+ 47us) client_negotiation.cc:240] Negotiation successful
0809 02:02:58.283262 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":425,"mutex_wait_us":1}
I20260809 02:02:58.284049 9938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:58.278218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.195:39329 (local address 127.0.0.1:43150)
0809 02:02:58.279066 (+ 848us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:58.279081 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 02:02:58.279254 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:02:58.279924 (+ 670us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:02:58.279926 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:02:58.279939 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:02:58.280029 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:58.280036 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:58.281602 (+ 1566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:58.281605 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:58.282691 (+ 1086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:58.282701 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:58.283495 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:58.283499 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:58.283714 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:58.283738 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 02:02:58.283832 (+ 94us) client_negotiation.cc:240] Negotiation successful
0809 02:02:58.283907 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":783,"mutex_wait_us":569,"thread_start_us":114,"threads_started":1}
I20260809 02:02:58.284109 9935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:58.278234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41758 (local address 127.1.192.193:37543)
0809 02:02:58.279035 (+ 801us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:58.279049 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:58.279215 (+ 166us) server_negotiation.cc:407] Connection header received
0809 02:02:58.279306 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:58.279313 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:58.279541 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:58.279722 (+ 181us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:02:58.280003 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:58.280789 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:58.282010 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:58.282884 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:58.283636 (+ 752us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:58.283792 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:58.283852 (+ 60us) server_negotiation.cc:299] Negotiation successful
0809 02:02:58.283958 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":672,"thread_start_us":94,"threads_started":1}
I20260809 02:02:58.284153 9937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:58.278460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43150 (local address 127.1.192.195:39329)
0809 02:02:58.278809 (+ 349us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:58.278825 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:58.279176 (+ 351us) server_negotiation.cc:407] Connection header received
0809 02:02:58.279412 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:58.279417 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:58.279616 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:58.279776 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:02:58.280157 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:58.281344 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:58.282810 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:58.283351 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:58.283489 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:58.283679 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:58.283866 (+ 187us) server_negotiation.cc:299] Negotiation successful
0809 02:02:58.283967 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":237,"thread_start_us":116,"threads_started":1}
I20260809 02:02:58.285640 9936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:58.278218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.194:35937 (local address 127.0.0.1:50332)
0809 02:02:58.278666 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:58.278681 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 02:02:58.278777 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:02:58.279780 (+ 1003us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:02:58.279783 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:02:58.279801 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:02:58.279931 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:58.279938 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:58.282418 (+ 2480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:58.282422 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:58.283730 (+ 1308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:58.283741 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:58.285214 (+ 1473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:58.285218 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:58.285310 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:58.285358 (+ 48us) client_negotiation.cc:769] Sending connection context
0809 02:02:58.285409 (+ 51us) client_negotiation.cc:240] Negotiation successful
0809 02:02:58.285509 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":350,"mutex_wait_us":23,"thread_start_us":138,"threads_started":1}
I20260809 02:02:58.286482 9939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:58.278979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50332 (local address 127.1.192.194:35937)
0809 02:02:58.279295 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:58.279308 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:58.279330 (+ 22us) server_negotiation.cc:407] Connection header received
0809 02:02:58.279408 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:58.279413 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:58.279616 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:58.280107 (+ 491us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:02:58.280252 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:58.282290 (+ 2038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:58.283873 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:58.285055 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:58.286048 (+ 993us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:58.286203 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:58.286254 (+ 51us) server_negotiation.cc:299] Negotiation successful
0809 02:02:58.286340 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":218,"thread_start_us":122,"threads_started":1}
I20260809 02:02:58.287050 9523 tablet_service.cc:1467] Processing CreateTablet for tablet 0fb6cb7eb28b4953951c77028d8cf4e5 (DEFAULT_TABLE table=test-table [id=6911c3e92ecb4ffc812f141d97142467]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:02:58.287039 9822 tablet_service.cc:1467] Processing CreateTablet for tablet 0fb6cb7eb28b4953951c77028d8cf4e5 (DEFAULT_TABLE table=test-table [id=6911c3e92ecb4ffc812f141d97142467]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:02:58.288045 9523 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0fb6cb7eb28b4953951c77028d8cf4e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:02:58.288089 9822 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0fb6cb7eb28b4953951c77028d8cf4e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:02:58.289117 9676 tablet_service.cc:1467] Processing CreateTablet for tablet 0fb6cb7eb28b4953951c77028d8cf4e5 (DEFAULT_TABLE table=test-table [id=6911c3e92ecb4ffc812f141d97142467]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:02:58.290153 9676 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0fb6cb7eb28b4953951c77028d8cf4e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:02:58.295025 9941 tablet_bootstrap.cc:492] T 0fb6cb7eb28b4953951c77028d8cf4e5 P 2eb7b1762aec408499a7ed56b7923dfc: Bootstrap starting.
I20260809 02:02:58.296890 9942 tablet_bootstrap.cc:492] T 0fb6cb7eb28b4953951c77028d8cf4e5 P cd2035d243e94eeea383bb2f79f51dc1: Bootstrap starting.
I20260809 02:02:58.297714 9941 tablet_bootstrap.cc:654] T 0fb6cb7eb28b4953951c77028d8cf4e5 P 2eb7b1762aec408499a7ed56b7923dfc: Neither blocks nor log segments found. Creating new log.
I20260809 02:02:58.297994 9943 tablet_bootstrap.cc:492] T 0fb6cb7eb28b4953951c77028d8cf4e5 P 8030853bc1144362a198609d5c2a1a6d: Bootstrap starting.
I20260809 02:02:58.298719 9941 log.cc:824] T 0fb6cb7eb28b4953951c77028d8cf4e5 P 2eb7b1762aec408499a7ed56b7923dfc: Log is configured to *not* fsync() on all Append() calls
I20260809 02:02:58.299206 9942 tablet_bootstrap.cc:654] T 0fb6cb7eb28b4953951c77028d8cf4e5 P cd2035d243e94eeea383bb2f79f51dc1: Neither blocks nor log segments found. Creating new log.
I20260809 02:02:58.299968 9942 log.cc:824] T 0fb6cb7eb28b4953951c77028d8cf4e5 P cd2035d243e94eeea383bb2f79f51dc1: Log is configured to *not* fsync() on all Append() calls
I20260809 02:02:58.300019 9943 tablet_bootstrap.cc:654] T 0fb6cb7eb28b4953951c77028d8cf4e5 P 8030853bc1144362a198609d5c2a1a6d: Neither blocks nor log segments found. Creating new log.
I20260809 02:02:58.300784 9943 log.cc:824] T 0fb6cb7eb28b4953951c77028d8cf4e5 P 8030853bc1144362a198609d5c2a1a6d: Log is configured to *not* fsync() on all Append() calls
I20260809 02:02:58.301061 9941 tablet_bootstrap.cc:492] T 0fb6cb7eb28b4953951c77028d8cf4e5 P 2eb7b1762aec408499a7ed56b7923dfc: No bootstrap required, opened a new log
I20260809 02:02:58.301292 9941 ts_tablet_manager.cc:1397] T 0fb6cb7eb28b4953951c77028d8cf4e5 P 2eb7b1762aec408499a7ed56b7923dfc: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260809 02:02:58.301848 9942 tablet_bootstrap.cc:492] T 0fb6cb7eb28b4953951c77028d8cf4e5 P cd2035d243e94eeea383bb2f79f51dc1: No bootstrap required, opened a new log
I20260809 02:02:58.302038 9942 ts_tablet_manager.cc:1397] T 0fb6cb7eb28b4953951c77028d8cf4e5 P cd2035d243e94eeea383bb2f79f51dc1: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260809 02:02:58.302770 9943 tablet_bootstrap.cc:492] T 0fb6cb7eb28b4953951c77028d8cf4e5 P 8030853bc1144362a198609d5c2a1a6d: No bootstrap required, opened a new log
I20260809 02:02:58.302934 9943 ts_tablet_manager.cc:1397] T 0fb6cb7eb28b4953951c77028d8cf4e5 P 8030853bc1144362a198609d5c2a1a6d: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260809 02:02:58.305729 9942 raft_consensus.cc:348] T 0fb6cb7eb28b4953951c77028d8cf4e5 P cd2035d243e94eeea383bb2f79f51dc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2eb7b1762aec408499a7ed56b7923dfc" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 37543 } } peers { permanent_uuid: "8030853bc1144362a198609d5c2a1a6d" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 35937 } } peers { permanent_uuid: "cd2035d243e94eeea383bb2f79f51dc1" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 39329 } }
I20260809 02:02:58.305974 9942 raft_consensus.cc:374] T 0fb6cb7eb28b4953951c77028d8cf4e5 P cd2035d243e94eeea383bb2f79f51dc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:02:58.306035 9942 raft_consensus.cc:729] T 0fb6cb7eb28b4953951c77028d8cf4e5 P cd2035d243e94eeea383bb2f79f51dc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd2035d243e94eeea383bb2f79f51dc1, State: Initialized, Role: FOLLOWER
I20260809 02:02:58.306493 9942 consensus_queue.cc:260] T 0fb6cb7eb28b4953951c77028d8cf4e5 P cd2035d243e94eeea383bb2f79f51dc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2eb7b1762aec408499a7ed56b7923dfc" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 37543 } } peers { permanent_uuid: "8030853bc1144362a198609d5c2a1a6d" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 35937 } } peers { permanent_uuid: "cd2035d243e94eeea383bb2f79f51dc1" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 39329 } }
I20260809 02:02:58.306476 9941 raft_consensus.cc:348] T 0fb6cb7eb28b4953951c77028d8cf4e5 P 2eb7b1762aec408499a7ed56b7923dfc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2eb7b1762aec408499a7ed56b7923dfc" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 37543 } } peers { permanent_uuid: "8030853bc1144362a198609d5c2a1a6d" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 35937 } } peers { permanent_uuid: "cd2035d243e94eeea383bb2f79f51dc1" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 39329 } }
I20260809 02:02:58.306805 9941 raft_consensus.cc:374] T 0fb6cb7eb28b4953951c77028d8cf4e5 P 2eb7b1762aec408499a7ed56b7923dfc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:02:58.307003 9941 raft_consensus.cc:729] T 0fb6cb7eb28b4953951c77028d8cf4e5 P 2eb7b1762aec408499a7ed56b7923dfc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2eb7b1762aec408499a7ed56b7923dfc, State: Initialized, Role: FOLLOWER
I20260809 02:02:58.307467 9942 ts_tablet_manager.cc:1428] T 0fb6cb7eb28b4953951c77028d8cf4e5 P cd2035d243e94eeea383bb2f79f51dc1: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260809 02:02:58.307456 9941 consensus_queue.cc:260] T 0fb6cb7eb28b4953951c77028d8cf4e5 P 2eb7b1762aec408499a7ed56b7923dfc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2eb7b1762aec408499a7ed56b7923dfc" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 37543 } } peers { permanent_uuid: "8030853bc1144362a198609d5c2a1a6d" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 35937 } } peers { permanent_uuid: "cd2035d243e94eeea383bb2f79f51dc1" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 39329 } }
I20260809 02:02:58.307483 9943 raft_consensus.cc:348] T 0fb6cb7eb28b4953951c77028d8cf4e5 P 8030853bc1144362a198609d5c2a1a6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2eb7b1762aec408499a7ed56b7923dfc" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 37543 } } peers { permanent_uuid: "8030853bc1144362a198609d5c2a1a6d" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 35937 } } peers { permanent_uuid: "cd2035d243e94eeea383bb2f79f51dc1" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 39329 } }
I20260809 02:02:58.307723 9943 raft_consensus.cc:374] T 0fb6cb7eb28b4953951c77028d8cf4e5 P 8030853bc1144362a198609d5c2a1a6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:02:58.307775 9943 raft_consensus.cc:729] T 0fb6cb7eb28b4953951c77028d8cf4e5 P 8030853bc1144362a198609d5c2a1a6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8030853bc1144362a198609d5c2a1a6d, State: Initialized, Role: FOLLOWER
I20260809 02:02:58.307875 9902 heartbeater.cc:503] Master 127.1.192.254:34543 was elected leader, sending a full tablet report...
I20260809 02:02:58.308202 9943 consensus_queue.cc:260] T 0fb6cb7eb28b4953951c77028d8cf4e5 P 8030853bc1144362a198609d5c2a1a6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2eb7b1762aec408499a7ed56b7923dfc" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 37543 } } peers { permanent_uuid: "8030853bc1144362a198609d5c2a1a6d" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 35937 } } peers { permanent_uuid: "cd2035d243e94eeea383bb2f79f51dc1" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 39329 } }
I20260809 02:02:58.309144 9943 ts_tablet_manager.cc:1428] T 0fb6cb7eb28b4953951c77028d8cf4e5 P 8030853bc1144362a198609d5c2a1a6d: Time spent starting tablet: real 0.006s user 0.000s sys 0.007s
I20260809 02:02:58.309360 9748 heartbeater.cc:503] Master 127.1.192.254:34543 was elected leader, sending a full tablet report...
I20260809 02:02:58.309497 9941 ts_tablet_manager.cc:1428] T 0fb6cb7eb28b4953951c77028d8cf4e5 P 2eb7b1762aec408499a7ed56b7923dfc: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20260809 02:02:58.309870 9602 heartbeater.cc:503] Master 127.1.192.254:34543 was elected leader, sending a full tablet report...
I20260809 02:02:58.433391 9950 raft_consensus.cc:482] T 0fb6cb7eb28b4953951c77028d8cf4e5 P 2eb7b1762aec408499a7ed56b7923dfc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:02:58.433676 9950 raft_consensus.cc:504] T 0fb6cb7eb28b4953951c77028d8cf4e5 P 2eb7b1762aec408499a7ed56b7923dfc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2eb7b1762aec408499a7ed56b7923dfc" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 37543 } } peers { permanent_uuid: "8030853bc1144362a198609d5c2a1a6d" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 35937 } } peers { permanent_uuid: "cd2035d243e94eeea383bb2f79f51dc1" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 39329 } }
I20260809 02:02:58.434933 9950 leader_election.cc:288] T 0fb6cb7eb28b4953951c77028d8cf4e5 P 2eb7b1762aec408499a7ed56b7923dfc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8030853bc1144362a198609d5c2a1a6d (127.1.192.194:35937), cd2035d243e94eeea383bb2f79f51dc1 (127.1.192.195:39329)
I20260809 02:02:58.439160 9937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:58.435730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.193:42065 (local address 127.1.192.195:39329)
0809 02:02:58.435866 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:58.435870 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:58.435882 (+ 12us) server_negotiation.cc:407] Connection header received
0809 02:02:58.435955 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:58.435958 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:58.436004 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:58.436088 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:02:58.436400 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:58.436944 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:58.438121 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:58.438502 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:58.438641 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:58.438697 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:58.438929 (+ 232us) server_negotiation.cc:299] Negotiation successful
0809 02:02:58.439007 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 02:02:58.439162 9939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:58.435328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.193:46287 (local address 127.1.192.194:35937)
0809 02:02:58.435500 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:58.435505 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:58.435522 (+ 17us) server_negotiation.cc:407] Connection header received
0809 02:02:58.435770 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:58.435777 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:58.435834 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:58.435931 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:02:58.436257 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:58.436837 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:58.438047 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:58.438494 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:58.438625 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:58.438680 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:58.438968 (+ 288us) server_negotiation.cc:299] Negotiation successful
0809 02:02:58.439050 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 02:02:58.439169 9952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:58.435311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.195:39329 (local address 127.1.192.193:42065)
0809 02:02:58.435726 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:58.435744 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 02:02:58.435833 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:02:58.436122 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:02:58.436125 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:02:58.436139 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:02:58.436276 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:58.436285 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:58.437069 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:58.437071 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:58.437961 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:58.437970 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:58.438648 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:58.438652 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:58.438783 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:58.438817 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 02:02:58.438879 (+ 62us) client_negotiation.cc:240] Negotiation successful
0809 02:02:58.438970 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"mutex_wait_us":49,"thread_start_us":95,"threads_started":1}
I20260809 02:02:58.439177 9951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:58.435145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.194:35937 (local address 127.1.192.193:46287)
0809 02:02:58.435430 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:58.435448 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 02:02:58.435561 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:02:58.435962 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:02:58.435965 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:02:58.435985 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:02:58.436135 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:58.436143 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:58.436965 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:58.436971 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:58.437926 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:02:58.437936 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:02:58.438638 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:02:58.438641 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:02:58.438815 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:58.438832 (+ 17us) client_negotiation.cc:769] Sending connection context
0809 02:02:58.438931 (+ 99us) client_negotiation.cc:240] Negotiation successful
0809 02:02:58.439007 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"mutex_wait_us":55,"thread_start_us":119,"threads_started":1}
I20260809 02:02:58.439839 9847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fb6cb7eb28b4953951c77028d8cf4e5" candidate_uuid: "2eb7b1762aec408499a7ed56b7923dfc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd2035d243e94eeea383bb2f79f51dc1" is_pre_election: true
I20260809 02:02:58.440042 9697 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fb6cb7eb28b4953951c77028d8cf4e5" candidate_uuid: "2eb7b1762aec408499a7ed56b7923dfc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8030853bc1144362a198609d5c2a1a6d" is_pre_election: true
I20260809 02:02:58.440191 9847 raft_consensus.cc:2454] T 0fb6cb7eb28b4953951c77028d8cf4e5 P cd2035d243e94eeea383bb2f79f51dc1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2eb7b1762aec408499a7ed56b7923dfc in term 0.
I20260809 02:02:58.440335 9697 raft_consensus.cc:2454] T 0fb6cb7eb28b4953951c77028d8cf4e5 P 8030853bc1144362a198609d5c2a1a6d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2eb7b1762aec408499a7ed56b7923dfc in term 0.
I20260809 02:02:58.440804 9469 leader_election.cc:302] T 0fb6cb7eb28b4953951c77028d8cf4e5 P 2eb7b1762aec408499a7ed56b7923dfc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2eb7b1762aec408499a7ed56b7923dfc, 8030853bc1144362a198609d5c2a1a6d, cd2035d243e94eeea383bb2f79f51dc1; no voters:
I20260809 02:02:58.441069 9950 raft_consensus.cc:2781] T 0fb6cb7eb28b4953951c77028d8cf4e5 P 2eb7b1762aec408499a7ed56b7923dfc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:02:58.441175 9950 raft_consensus.cc:482] T 0fb6cb7eb28b4953951c77028d8cf4e5 P 2eb7b1762aec408499a7ed56b7923dfc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:02:58.441234 9950 raft_consensus.cc:3037] T 0fb6cb7eb28b4953951c77028d8cf4e5 P 2eb7b1762aec408499a7ed56b7923dfc [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:02:58.442416 9950 raft_consensus.cc:504] T 0fb6cb7eb28b4953951c77028d8cf4e5 P 2eb7b1762aec408499a7ed56b7923dfc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2eb7b1762aec408499a7ed56b7923dfc" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 37543 } } peers { permanent_uuid: "8030853bc1144362a198609d5c2a1a6d" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 35937 } } peers { permanent_uuid: "cd2035d243e94eeea383bb2f79f51dc1" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 39329 } }
I20260809 02:02:58.442823 9950 leader_election.cc:288] T 0fb6cb7eb28b4953951c77028d8cf4e5 P 2eb7b1762aec408499a7ed56b7923dfc [CANDIDATE]: Term 1 election: Requested vote from peers 8030853bc1144362a198609d5c2a1a6d (127.1.192.194:35937), cd2035d243e94eeea383bb2f79f51dc1 (127.1.192.195:39329)
I20260809 02:02:58.443225 9847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fb6cb7eb28b4953951c77028d8cf4e5" candidate_uuid: "2eb7b1762aec408499a7ed56b7923dfc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd2035d243e94eeea383bb2f79f51dc1"
I20260809 02:02:58.443260 9697 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fb6cb7eb28b4953951c77028d8cf4e5" candidate_uuid: "2eb7b1762aec408499a7ed56b7923dfc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8030853bc1144362a198609d5c2a1a6d"
I20260809 02:02:58.443359 9847 raft_consensus.cc:3037] T 0fb6cb7eb28b4953951c77028d8cf4e5 P cd2035d243e94eeea383bb2f79f51dc1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:02:58.443416 9697 raft_consensus.cc:3037] T 0fb6cb7eb28b4953951c77028d8cf4e5 P 8030853bc1144362a198609d5c2a1a6d [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:02:58.444551 9847 raft_consensus.cc:2454] T 0fb6cb7eb28b4953951c77028d8cf4e5 P cd2035d243e94eeea383bb2f79f51dc1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2eb7b1762aec408499a7ed56b7923dfc in term 1.
I20260809 02:02:58.444696 9697 raft_consensus.cc:2454] T 0fb6cb7eb28b4953951c77028d8cf4e5 P 8030853bc1144362a198609d5c2a1a6d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2eb7b1762aec408499a7ed56b7923dfc in term 1.
I20260809 02:02:58.444905 9472 leader_election.cc:302] T 0fb6cb7eb28b4953951c77028d8cf4e5 P 2eb7b1762aec408499a7ed56b7923dfc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2eb7b1762aec408499a7ed56b7923dfc, cd2035d243e94eeea383bb2f79f51dc1; no voters:
I20260809 02:02:58.445145 9950 raft_consensus.cc:2781] T 0fb6cb7eb28b4953951c77028d8cf4e5 P 2eb7b1762aec408499a7ed56b7923dfc [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:02:58.445389 9950 raft_consensus.cc:686] T 0fb6cb7eb28b4953951c77028d8cf4e5 P 2eb7b1762aec408499a7ed56b7923dfc [term 1 LEADER]: Becoming Leader. State: Replica: 2eb7b1762aec408499a7ed56b7923dfc, State: Running, Role: LEADER
I20260809 02:02:58.445807 9950 consensus_queue.cc:237] T 0fb6cb7eb28b4953951c77028d8cf4e5 P 2eb7b1762aec408499a7ed56b7923dfc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2eb7b1762aec408499a7ed56b7923dfc" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 37543 } } peers { permanent_uuid: "8030853bc1144362a198609d5c2a1a6d" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 35937 } } peers { permanent_uuid: "cd2035d243e94eeea383bb2f79f51dc1" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 39329 } }
I20260809 02:02:58.449674 8527 catalog_manager.cc:5495] T 0fb6cb7eb28b4953951c77028d8cf4e5 P 2eb7b1762aec408499a7ed56b7923dfc reported cstate change: term changed from 0 to 1, leader changed from <none> to 2eb7b1762aec408499a7ed56b7923dfc (127.1.192.193). New cstate: current_term: 1 leader_uuid: "2eb7b1762aec408499a7ed56b7923dfc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2eb7b1762aec408499a7ed56b7923dfc" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 37543 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8030853bc1144362a198609d5c2a1a6d" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 35937 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd2035d243e94eeea383bb2f79f51dc1" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 39329 } health_report { overall_health: UNKNOWN } } }
I20260809 02:02:58.475410 9935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:58.471260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41774 (local address 127.1.192.193:37543)
0809 02:02:58.471395 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:58.471398 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:58.471411 (+ 13us) server_negotiation.cc:407] Connection header received
0809 02:02:58.471494 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:58.471496 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:58.471552 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:58.471651 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:02:58.471911 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:58.472418 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:58.473583 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:58.473918 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:58.474056 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:58.474378 (+ 322us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:02:58.474843 (+ 465us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:02:58.474939 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:58.475096 (+ 157us) server_negotiation.cc:299] Negotiation successful
0809 02:02:58.475185 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
W20260809 02:02:58.480086 9905 tablet.cc:2369] T 0fb6cb7eb28b4953951c77028d8cf4e5 P cd2035d243e94eeea383bb2f79f51dc1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 02:02:58.481815 9847 raft_consensus.cc:1261] T 0fb6cb7eb28b4953951c77028d8cf4e5 P cd2035d243e94eeea383bb2f79f51dc1 [term 1 FOLLOWER]: Refusing update from remote peer 2eb7b1762aec408499a7ed56b7923dfc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:02:58.481850 9697 raft_consensus.cc:1261] T 0fb6cb7eb28b4953951c77028d8cf4e5 P 8030853bc1144362a198609d5c2a1a6d [term 1 FOLLOWER]: Refusing update from remote peer 2eb7b1762aec408499a7ed56b7923dfc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:02:58.482687 9953 consensus_queue.cc:1035] T 0fb6cb7eb28b4953951c77028d8cf4e5 P 2eb7b1762aec408499a7ed56b7923dfc [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd2035d243e94eeea383bb2f79f51dc1" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 39329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:02:58.482909 9950 consensus_queue.cc:1035] T 0fb6cb7eb28b4953951c77028d8cf4e5 P 2eb7b1762aec408499a7ed56b7923dfc [LEADER]: Connected to new peer: Peer: permanent_uuid: "8030853bc1144362a198609d5c2a1a6d" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 35937 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:02:58.503384 8526 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60490:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:02:58.503595 8526 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60490:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:02:58.508889 8526 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 14381b53c95b4bda8c4273431f4a1762: Sending DeleteTablet for 3 replicas of tablet 0fb6cb7eb28b4953951c77028d8cf4e5
I20260809 02:02:58.509653 9523 tablet_service.cc:1514] Processing DeleteTablet for tablet 0fb6cb7eb28b4953951c77028d8cf4e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:02:58 UTC) from {username='oryx', principal='oryx/127.1.192.254@KRBTEST.COM'} at 127.0.0.1:41758
I20260809 02:02:58.509802 9676 tablet_service.cc:1514] Processing DeleteTablet for tablet 0fb6cb7eb28b4953951c77028d8cf4e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:02:58 UTC) from {username='oryx', principal='oryx/127.1.192.254@KRBTEST.COM'} at 127.0.0.1:50332
I20260809 02:02:58.509923 9822 tablet_service.cc:1514] Processing DeleteTablet for tablet 0fb6cb7eb28b4953951c77028d8cf4e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:02:58 UTC) from {username='oryx', principal='oryx/127.1.192.254@KRBTEST.COM'} at 127.0.0.1:43150
I20260809 02:02:58.510362 9969 tablet_replica.cc:330] T 0fb6cb7eb28b4953951c77028d8cf4e5 P cd2035d243e94eeea383bb2f79f51dc1: stopping tablet replica
I20260809 02:02:58.510356 9967 tablet_replica.cc:330] T 0fb6cb7eb28b4953951c77028d8cf4e5 P 2eb7b1762aec408499a7ed56b7923dfc: stopping tablet replica
I20260809 02:02:58.510707 9969 raft_consensus.cc:2229] T 0fb6cb7eb28b4953951c77028d8cf4e5 P cd2035d243e94eeea383bb2f79f51dc1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:02:58.511358 9968 tablet_replica.cc:330] T 0fb6cb7eb28b4953951c77028d8cf4e5 P 8030853bc1144362a198609d5c2a1a6d: stopping tablet replica
I20260809 02:02:58.511662 9968 raft_consensus.cc:2229] T 0fb6cb7eb28b4953951c77028d8cf4e5 P 8030853bc1144362a198609d5c2a1a6d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:02:58.511866 9967 raft_consensus.cc:2229] T 0fb6cb7eb28b4953951c77028d8cf4e5 P 2eb7b1762aec408499a7ed56b7923dfc [term 1 LEADER]: Raft consensus shutting down.
I20260809 02:02:58.511973 9968 raft_consensus.cc:2258] T 0fb6cb7eb28b4953951c77028d8cf4e5 P 8030853bc1144362a198609d5c2a1a6d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:02:58.512008 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 8491
I20260809 02:02:58.514660 9967 raft_consensus.cc:2258] T 0fb6cb7eb28b4953951c77028d8cf4e5 P 2eb7b1762aec408499a7ed56b7923dfc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:02:58.515021 9969 raft_consensus.cc:2258] T 0fb6cb7eb28b4953951c77028d8cf4e5 P cd2035d243e94eeea383bb2f79f51dc1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:02:58.516409 9967 ts_tablet_manager.cc:1905] T 0fb6cb7eb28b4953951c77028d8cf4e5 P 2eb7b1762aec408499a7ed56b7923dfc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:02:58.516568 9968 ts_tablet_manager.cc:1905] T 0fb6cb7eb28b4953951c77028d8cf4e5 P 8030853bc1144362a198609d5c2a1a6d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:02:58.519376 9967 ts_tablet_manager.cc:1918] T 0fb6cb7eb28b4953951c77028d8cf4e5 P 2eb7b1762aec408499a7ed56b7923dfc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:02:58.519477 9967 log.cc:1196] T 0fb6cb7eb28b4953951c77028d8cf4e5 P 2eb7b1762aec408499a7ed56b7923dfc: Deleting WAL directory at /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/ts-0/wal/wals/0fb6cb7eb28b4953951c77028d8cf4e5
I20260809 02:02:58.519519 9968 ts_tablet_manager.cc:1918] T 0fb6cb7eb28b4953951c77028d8cf4e5 P 8030853bc1144362a198609d5c2a1a6d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:02:58.519605 9968 log.cc:1196] T 0fb6cb7eb28b4953951c77028d8cf4e5 P 8030853bc1144362a198609d5c2a1a6d: Deleting WAL directory at /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/ts-1/wal/wals/0fb6cb7eb28b4953951c77028d8cf4e5
I20260809 02:02:58.519851 9967 ts_tablet_manager.cc:1939] T 0fb6cb7eb28b4953951c77028d8cf4e5 P 2eb7b1762aec408499a7ed56b7923dfc: Deleting consensus metadata
I20260809 02:02:58.519906 9968 ts_tablet_manager.cc:1939] T 0fb6cb7eb28b4953951c77028d8cf4e5 P 8030853bc1144362a198609d5c2a1a6d: Deleting consensus metadata
W20260809 02:02:58.522874 9630 connection.cc:535] server connection from 127.0.0.1:50332 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260809 02:02:58.522908 9470 connection.cc:535] server connection from 127.0.0.1:41758 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 02:02:58.524129 9969 ts_tablet_manager.cc:1905] T 0fb6cb7eb28b4953951c77028d8cf4e5 P cd2035d243e94eeea383bb2f79f51dc1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:02:58.527117 9969 ts_tablet_manager.cc:1918] T 0fb6cb7eb28b4953951c77028d8cf4e5 P cd2035d243e94eeea383bb2f79f51dc1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:02:58.527228 9969 log.cc:1196] T 0fb6cb7eb28b4953951c77028d8cf4e5 P cd2035d243e94eeea383bb2f79f51dc1: Deleting WAL directory at /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/ts-2/wal/wals/0fb6cb7eb28b4953951c77028d8cf4e5
I20260809 02:02:58.527652 9969 ts_tablet_manager.cc:1939] T 0fb6cb7eb28b4953951c77028d8cf4e5 P cd2035d243e94eeea383bb2f79f51dc1: Deleting consensus metadata
W20260809 02:02:58.529989 9776 connection.cc:441] server connection from 127.0.0.1:43150 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43150 (request call id 1) could send its response
I20260809 02:02:58.532256 9919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:58.511522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48604 (local address 127.1.192.253:40841)
0809 02:02:58.511636 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:58.511640 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:58.521285 (+ 9645us) server_negotiation.cc:407] Connection header received
0809 02:02:58.521443 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:58.521446 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:58.521526 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:58.521613 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:58.523342 (+ 1729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:58.524147 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:58.525316 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:58.525717 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:58.525802 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:58.528057 (+ 2255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:02:58.528060 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:02:58.528063 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:02:58.528090 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:02:58.530332 (+ 2242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:58.530774 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:58.530780 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:58.530782 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:58.530867 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:58.531177 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:58.531180 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:58.531182 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:58.531436 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:02:58.531593 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:58.531946 (+ 353us) server_negotiation.cc:299] Negotiation successful
0809 02:02:58.532084 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 02:02:58.541399 9914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:58.511080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32876 (local address 127.1.192.252:44581)
0809 02:02:58.511190 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:58.511194 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:58.532360 (+ 21166us) server_negotiation.cc:407] Connection header received
0809 02:02:58.532555 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:58.532559 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:58.532610 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:58.532669 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:58.533556 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:58.534365 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:58.535186 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:58.535540 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:58.535640 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:58.537369 (+ 1729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:02:58.537374 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:02:58.537378 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:02:58.537410 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:02:58.539521 (+ 2111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:58.540007 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:58.540011 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:58.540013 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:58.540116 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:58.540425 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:58.540429 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:58.540431 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:58.540639 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:02:58.540740 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:58.541089 (+ 349us) server_negotiation.cc:299] Negotiation successful
0809 02:02:58.541231 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 02:02:58.542817 9605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:58.531519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32880 (local address 127.1.192.252:44581)
0809 02:02:58.531648 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:58.531653 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:58.531958 (+ 305us) server_negotiation.cc:407] Connection header received
0809 02:02:58.532168 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:58.532171 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:58.532228 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:58.532283 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:58.533715 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:58.534399 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:58.535866 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:58.536180 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:58.536287 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:58.537835 (+ 1548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:02:58.537842 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:02:58.537849 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:02:58.537883 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:02:58.540977 (+ 3094us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:58.541440 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:58.541443 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:58.541445 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:58.541553 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:58.541853 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:58.541857 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:58.541859 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:58.542072 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:02:58.542191 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:58.542549 (+ 358us) server_negotiation.cc:299] Negotiation successful
0809 02:02:58.542665 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 02:02:58.548226 9919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:58.538260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48620 (local address 127.1.192.253:40841)
0809 02:02:58.538361 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0809 02:02:58.538364 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:02:58.538654 (+ 290us) server_negotiation.cc:407] Connection header received
0809 02:02:58.538786 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:02:58.538789 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:02:58.538833 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:02:58.538905 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:02:58.539696 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:58.540540 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:58.541858 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:02:58.542191 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:02:58.542268 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:02:58.544462 (+ 2194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:02:58.544466 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:02:58.544469 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:02:58.544494 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:02:58.546378 (+ 1884us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:58.546888 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:58.546891 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:58.546893 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:58.546984 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:02:58.547240 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:02:58.547243 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:02:58.547245 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:02:58.547464 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:02:58.547592 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0809 02:02:58.547932 (+ 340us) server_negotiation.cc:299] Negotiation successful
0809 02:02:58.548053 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 02:02:59.495570 9977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:59.494976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:34543 (local address 127.1.192.194:43189)
0809 02:02:59.495336 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:59.495446 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:34543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":130,"threads_started":1}
W20260809 02:02:59.495934 9748 heartbeater.cc:644] Failed to heartbeat to 127.1.192.254:34543 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.1.192.254:34543: connect: Connection refused (error 111)
I20260809 02:02:59.497702 9978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:59.497185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:34543 (local address 127.1.192.193:53071)
0809 02:02:59.497437 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:59.497581 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:34543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":102,"threads_started":1}
W20260809 02:02:59.498037 9602 heartbeater.cc:644] Failed to heartbeat to 127.1.192.254:34543 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.1.192.254:34543: connect: Connection refused (error 111)
I20260809 02:02:59.503690 9979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:02:59.503147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:34543 (local address 127.1.192.195:45929)
0809 02:02:59.503471 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0809 02:02:59.503575 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:34543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":92,"threads_started":1}
W20260809 02:02:59.504065 9902 heartbeater.cc:644] Failed to heartbeat to 127.1.192.254:34543 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.1.192.254:34543: connect: Connection refused (error 111)
I20260809 02:03:00.008526 9980 raft_consensus.cc:482] T 00000000000000000000000000000000 P d909896773e04b93bbb1e1e7d4c11264 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 14381b53c95b4bda8c4273431f4a1762)
I20260809 02:03:00.008738 9980 raft_consensus.cc:504] T 00000000000000000000000000000000 P d909896773e04b93bbb1e1e7d4c11264 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "14381b53c95b4bda8c4273431f4a1762" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 34543 } } peers { permanent_uuid: "bf6ddc1ba97745d897049f1300717b5b" member_type: VOTER last_known_addr { host: "127.1.192.253" port: 40841 } attrs { promote: false } } peers { permanent_uuid: "d909896773e04b93bbb1e1e7d4c11264" member_type: VOTER last_known_addr { host: "127.1.192.252" port: 44581 } attrs { promote: false } }
I20260809 02:03:00.009294 9981 raft_consensus.cc:482] T 00000000000000000000000000000000 P bf6ddc1ba97745d897049f1300717b5b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 14381b53c95b4bda8c4273431f4a1762)
I20260809 02:03:00.009446 9981 raft_consensus.cc:504] T 00000000000000000000000000000000 P bf6ddc1ba97745d897049f1300717b5b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "14381b53c95b4bda8c4273431f4a1762" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 34543 } } peers { permanent_uuid: "bf6ddc1ba97745d897049f1300717b5b" member_type: VOTER last_known_addr { host: "127.1.192.253" port: 40841 } attrs { promote: false } } peers { permanent_uuid: "d909896773e04b93bbb1e1e7d4c11264" member_type: VOTER last_known_addr { host: "127.1.192.252" port: 44581 } attrs { promote: false } }
I20260809 02:03:00.009915 9980 leader_election.cc:288] T 00000000000000000000000000000000 P d909896773e04b93bbb1e1e7d4c11264 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 14381b53c95b4bda8c4273431f4a1762 (127.1.192.254:34543), bf6ddc1ba97745d897049f1300717b5b (127.1.192.253:40841)
I20260809 02:03:00.010097 9981 leader_election.cc:288] T 00000000000000000000000000000000 P bf6ddc1ba97745d897049f1300717b5b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 14381b53c95b4bda8c4273431f4a1762 (127.1.192.254:34543), d909896773e04b93bbb1e1e7d4c11264 (127.1.192.252:44581)
I20260809 02:03:00.010658 9982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:00.010148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:34543 (local address 127.0.0.1:60888)
0809 02:03:00.010449 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:00.010540 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:34543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":88,"threads_started":1}
W20260809 02:03:00.011034 9380 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.192.254:34543: connect: Connection refused (error 111)
I20260809 02:03:00.011278 9983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:00.010285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:34543 (local address 127.0.0.1:60900)
0809 02:03:00.011038 (+ 753us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:00.011123 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:34543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":704,"mutex_wait_us":278,"thread_start_us":49,"threads_started":1}
W20260809 02:03:00.011538 9179 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.192.254:34543: connect: Connection refused (error 111)
I20260809 02:03:00.011983 9982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:00.011737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:34543 (local address 127.0.0.1:60912)
0809 02:03:00.011869 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:00.011916 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:34543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
W20260809 02:03:00.012135 9380 leader_election.cc:334] T 00000000000000000000000000000000 P d909896773e04b93bbb1e1e7d4c11264 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 14381b53c95b4bda8c4273431f4a1762 (127.1.192.254:34543): Network error: Client connection negotiation failed: client connection to 127.1.192.254:34543: connect: Connection refused (error 111)
I20260809 02:03:00.012693 9983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:00.012433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:34543 (local address 127.0.0.1:60924)
0809 02:03:00.012562 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:00.012628 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:34543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260809 02:03:00.012816 9179 leader_election.cc:334] T 00000000000000000000000000000000 P bf6ddc1ba97745d897049f1300717b5b [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 14381b53c95b4bda8c4273431f4a1762 (127.1.192.254:34543): Network error: Client connection negotiation failed: client connection to 127.1.192.254:34543: connect: Connection refused (error 111)
I20260809 02:03:00.015136 9985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:00.010490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.253:40841 (local address 127.0.0.1:48636)
0809 02:03:00.010681 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:00.010696 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 02:03:00.010808 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:03:00.011291 (+ 483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:03:00.011296 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:03:00.011319 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:03:00.011478 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:00.011487 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:00.012651 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:00.012655 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:00.013771 (+ 1116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:00.013781 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:00.014600 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:00.014604 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:00.014807 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:00.014835 (+ 28us) client_negotiation.cc:769] Sending connection context
0809 02:03:00.014919 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 02:03:00.014997 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"mutex_wait_us":25,"spinlock_wait_cycles":1792,"thread_start_us":56,"threads_started":1}
I20260809 02:03:00.015177 9987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:00.010761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48636 (local address 127.1.192.253:40841)
0809 02:03:00.011029 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0809 02:03:00.011033 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:03:00.011049 (+ 16us) server_negotiation.cc:407] Connection header received
0809 02:03:00.011100 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:03:00.011103 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:03:00.011153 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:03:00.011610 (+ 457us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:03:00.011726 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:00.012502 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:00.013892 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:00.014473 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:00.014580 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:00.014638 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 02:03:00.014957 (+ 319us) server_negotiation.cc:299] Negotiation successful
0809 02:03:00.015042 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":203,"thread_start_us":64,"threads_started":1}
I20260809 02:03:00.015588 9204 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d909896773e04b93bbb1e1e7d4c11264" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "bf6ddc1ba97745d897049f1300717b5b" is_pre_election: true
I20260809 02:03:00.015738 9986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:00.010282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.252:44581 (local address 127.0.0.1:32888)
0809 02:03:00.010904 (+ 622us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:00.010921 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 02:03:00.011030 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:03:00.011454 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:03:00.011457 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:03:00.011474 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:03:00.011575 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:00.011581 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:00.012616 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:00.012623 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:00.014171 (+ 1548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:00.014185 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:00.015125 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:00.015128 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:00.015218 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:00.015459 (+ 241us) client_negotiation.cc:769] Sending connection context
0809 02:03:00.015535 (+ 76us) client_negotiation.cc:240] Negotiation successful
0809 02:03:00.015617 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":550,"thread_start_us":72,"threads_started":1}
I20260809 02:03:00.015805 9204 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bf6ddc1ba97745d897049f1300717b5b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d909896773e04b93bbb1e1e7d4c11264 in term 1.
I20260809 02:03:00.015990 9984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:00.010465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32888 (local address 127.1.192.252:44581)
0809 02:03:00.010923 (+ 458us) server_negotiation.cc:206] Beginning negotiation
0809 02:03:00.010930 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 02:03:00.011153 (+ 223us) server_negotiation.cc:407] Connection header received
0809 02:03:00.011208 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:03:00.011211 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:03:00.011256 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:03:00.011339 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:03:00.011741 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:00.012519 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:00.014303 (+ 1784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:00.014910 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:00.015017 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:00.015069 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 02:03:00.015572 (+ 503us) server_negotiation.cc:299] Negotiation successful
0809 02:03:00.015653 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":295,"server-negotiator.queue_time_us":399,"spinlock_wait_cycles":768,"thread_start_us":63,"threads_started":1}
I20260809 02:03:00.016140 9380 leader_election.cc:302] T 00000000000000000000000000000000 P d909896773e04b93bbb1e1e7d4c11264 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bf6ddc1ba97745d897049f1300717b5b, d909896773e04b93bbb1e1e7d4c11264; no voters: 14381b53c95b4bda8c4273431f4a1762
I20260809 02:03:00.016367 9980 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d909896773e04b93bbb1e1e7d4c11264 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 02:03:00.016455 9980 raft_consensus.cc:482] T 00000000000000000000000000000000 P d909896773e04b93bbb1e1e7d4c11264 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 14381b53c95b4bda8c4273431f4a1762)
I20260809 02:03:00.016381 9412 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bf6ddc1ba97745d897049f1300717b5b" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "d909896773e04b93bbb1e1e7d4c11264" is_pre_election: true
I20260809 02:03:00.016498 9980 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d909896773e04b93bbb1e1e7d4c11264 [term 1 FOLLOWER]: Advancing to term 2
I20260809 02:03:00.017681 9980 raft_consensus.cc:504] T 00000000000000000000000000000000 P d909896773e04b93bbb1e1e7d4c11264 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "14381b53c95b4bda8c4273431f4a1762" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 34543 } } peers { permanent_uuid: "bf6ddc1ba97745d897049f1300717b5b" member_type: VOTER last_known_addr { host: "127.1.192.253" port: 40841 } attrs { promote: false } } peers { permanent_uuid: "d909896773e04b93bbb1e1e7d4c11264" member_type: VOTER last_known_addr { host: "127.1.192.252" port: 44581 } attrs { promote: false } }
I20260809 02:03:00.017952 9412 raft_consensus.cc:2379] T 00000000000000000000000000000000 P d909896773e04b93bbb1e1e7d4c11264 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bf6ddc1ba97745d897049f1300717b5b in current term 2: Already voted for candidate d909896773e04b93bbb1e1e7d4c11264 in this term.
I20260809 02:03:00.018105 9980 leader_election.cc:288] T 00000000000000000000000000000000 P d909896773e04b93bbb1e1e7d4c11264 [CANDIDATE]: Term 2 election: Requested vote from peers 14381b53c95b4bda8c4273431f4a1762 (127.1.192.254:34543), bf6ddc1ba97745d897049f1300717b5b (127.1.192.253:40841)
I20260809 02:03:00.018407 9985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:00.018205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:34543 (local address 127.0.0.1:60934)
0809 02:03:00.018285 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:00.018339 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:34543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260809 02:03:00.018558 9204 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d909896773e04b93bbb1e1e7d4c11264" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "bf6ddc1ba97745d897049f1300717b5b"
I20260809 02:03:00.018682 9204 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bf6ddc1ba97745d897049f1300717b5b [term 1 FOLLOWER]: Advancing to term 2
I20260809 02:03:00.018694 9181 leader_election.cc:302] T 00000000000000000000000000000000 P bf6ddc1ba97745d897049f1300717b5b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bf6ddc1ba97745d897049f1300717b5b; no voters: 14381b53c95b4bda8c4273431f4a1762, d909896773e04b93bbb1e1e7d4c11264
I20260809 02:03:00.019636 9204 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bf6ddc1ba97745d897049f1300717b5b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d909896773e04b93bbb1e1e7d4c11264 in term 2.
I20260809 02:03:00.019770 9981 raft_consensus.cc:2726] T 00000000000000000000000000000000 P bf6ddc1ba97745d897049f1300717b5b [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 02:03:00.020181 9380 leader_election.cc:302] T 00000000000000000000000000000000 P d909896773e04b93bbb1e1e7d4c11264 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bf6ddc1ba97745d897049f1300717b5b, d909896773e04b93bbb1e1e7d4c11264; no voters:
I20260809 02:03:00.020354 9980 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d909896773e04b93bbb1e1e7d4c11264 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 02:03:00.020576 9980 raft_consensus.cc:686] T 00000000000000000000000000000000 P d909896773e04b93bbb1e1e7d4c11264 [term 2 LEADER]: Becoming Leader. State: Replica: d909896773e04b93bbb1e1e7d4c11264, State: Running, Role: LEADER
I20260809 02:03:00.020695 9985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:00.020444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:34543 (local address 127.0.0.1:60944)
0809 02:03:00.020587 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:00.020642 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:34543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
W20260809 02:03:00.020769 9380 leader_election.cc:334] T 00000000000000000000000000000000 P d909896773e04b93bbb1e1e7d4c11264 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 14381b53c95b4bda8c4273431f4a1762 (127.1.192.254:34543): Network error: Client connection negotiation failed: client connection to 127.1.192.254:34543: connect: Connection refused (error 111)
I20260809 02:03:00.020864 9980 consensus_queue.cc:237] T 00000000000000000000000000000000 P d909896773e04b93bbb1e1e7d4c11264 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "14381b53c95b4bda8c4273431f4a1762" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 34543 } } peers { permanent_uuid: "bf6ddc1ba97745d897049f1300717b5b" member_type: VOTER last_known_addr { host: "127.1.192.253" port: 40841 } attrs { promote: false } } peers { permanent_uuid: "d909896773e04b93bbb1e1e7d4c11264" member_type: VOTER last_known_addr { host: "127.1.192.252" port: 44581 } attrs { promote: false } }
I20260809 02:03:00.021907 9991 sys_catalog.cc:455] T 00000000000000000000000000000000 P d909896773e04b93bbb1e1e7d4c11264 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d909896773e04b93bbb1e1e7d4c11264. Latest consensus state: current_term: 2 leader_uuid: "d909896773e04b93bbb1e1e7d4c11264" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "14381b53c95b4bda8c4273431f4a1762" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 34543 } } peers { permanent_uuid: "bf6ddc1ba97745d897049f1300717b5b" member_type: VOTER last_known_addr { host: "127.1.192.253" port: 40841 } attrs { promote: false } } peers { permanent_uuid: "d909896773e04b93bbb1e1e7d4c11264" member_type: VOTER last_known_addr { host: "127.1.192.252" port: 44581 } attrs { promote: false } } }
I20260809 02:03:00.022019 9991 sys_catalog.cc:458] T 00000000000000000000000000000000 P d909896773e04b93bbb1e1e7d4c11264 [sys.catalog]: This master's current role is: LEADER
I20260809 02:03:00.022306 9993 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:03:00.024693 9993 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:03:00.025036 9993 catalog_manager.cc:1259] Loaded cluster ID: 279e275c39804b08b8c01de18b06bcb9
I20260809 02:03:00.025080 9993 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:03:00.025676 9993 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:03:00.026014 9993 catalog_manager.cc:5879] T 00000000000000000000000000000000 P d909896773e04b93bbb1e1e7d4c11264: Loaded TSK: 0
I20260809 02:03:00.026714 9993 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:03:00.077263 9397 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32880:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 02:03:00.080466 9397 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:03:00.083633 9985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:00.083375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:34543 (local address 127.0.0.1:60962)
0809 02:03:00.083511 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:00.083566 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:34543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260809 02:03:00.083663 9204 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bf6ddc1ba97745d897049f1300717b5b [term 2 FOLLOWER]: Refusing update from remote peer d909896773e04b93bbb1e1e7d4c11264: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260809 02:03:00.084182 9980 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d909896773e04b93bbb1e1e7d4c11264 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf6ddc1ba97745d897049f1300717b5b" member_type: VOTER last_known_addr { host: "127.1.192.253" port: 40841 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260809 02:03:00.084522 9985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:00.084278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:34543 (local address 127.0.0.1:60968)
0809 02:03:00.084410 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:00.084468 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:34543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260809 02:03:00.084726 9380 consensus_peers.cc:487] T 00000000000000000000000000000000 P d909896773e04b93bbb1e1e7d4c11264 -> Peer 14381b53c95b4bda8c4273431f4a1762 (127.1.192.254:34543): Couldn't send request to peer 14381b53c95b4bda8c4273431f4a1762. Status: Network error: Client connection negotiation failed: client connection to 127.1.192.254:34543: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260809 02:03:00.085768 9981 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf6ddc1ba97745d897049f1300717b5b [sys.catalog]: SysCatalogTable state changed. Reason: New leader d909896773e04b93bbb1e1e7d4c11264. Latest consensus state: current_term: 2 leader_uuid: "d909896773e04b93bbb1e1e7d4c11264" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "14381b53c95b4bda8c4273431f4a1762" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 34543 } } peers { permanent_uuid: "bf6ddc1ba97745d897049f1300717b5b" member_type: VOTER last_known_addr { host: "127.1.192.253" port: 40841 } attrs { promote: false } } peers { permanent_uuid: "d909896773e04b93bbb1e1e7d4c11264" member_type: VOTER last_known_addr { host: "127.1.192.252" port: 44581 } attrs { promote: false } } }
I20260809 02:03:00.085899 9981 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf6ddc1ba97745d897049f1300717b5b [sys.catalog]: This master's current role is: FOLLOWER
I20260809 02:03:00.087037 9980 sys_catalog.cc:455] T 00000000000000000000000000000000 P d909896773e04b93bbb1e1e7d4c11264 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "d909896773e04b93bbb1e1e7d4c11264" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "14381b53c95b4bda8c4273431f4a1762" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 34543 } } peers { permanent_uuid: "bf6ddc1ba97745d897049f1300717b5b" member_type: VOTER last_known_addr { host: "127.1.192.253" port: 40841 } attrs { promote: false } } peers { permanent_uuid: "d909896773e04b93bbb1e1e7d4c11264" member_type: VOTER last_known_addr { host: "127.1.192.252" port: 44581 } attrs { promote: false } } }
I20260809 02:03:00.087196 9980 sys_catalog.cc:458] T 00000000000000000000000000000000 P d909896773e04b93bbb1e1e7d4c11264 [sys.catalog]: This master's current role is: LEADER
I20260809 02:03:00.087215 9995 mvcc.cc:205] Tried to move back new op lower bound from 7316443054417772544 to 7316443054168133632. Current Snapshot: MvccSnapshot[applied={T|T < 7316443054417772544}]
I20260809 02:03:00.087634 9981 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf6ddc1ba97745d897049f1300717b5b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "d909896773e04b93bbb1e1e7d4c11264" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "14381b53c95b4bda8c4273431f4a1762" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 34543 } } peers { permanent_uuid: "bf6ddc1ba97745d897049f1300717b5b" member_type: VOTER last_known_addr { host: "127.1.192.253" port: 40841 } attrs { promote: false } } peers { permanent_uuid: "d909896773e04b93bbb1e1e7d4c11264" member_type: VOTER last_known_addr { host: "127.1.192.252" port: 44581 } attrs { promote: false } } }
I20260809 02:03:00.087757 9981 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf6ddc1ba97745d897049f1300717b5b [sys.catalog]: This master's current role is: FOLLOWER
I20260809 02:03:00.097908 9982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:00.093308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.195:39329 (local address 127.0.0.1:43160)
0809 02:03:00.094009 (+ 701us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:00.094021 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:03:00.094157 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:03:00.094439 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:03:00.094445 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:03:00.094467 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:03:00.094575 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:00.094582 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:00.095663 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:00.095667 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:00.096577 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:00.096584 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:00.097450 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:00.097453 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:00.097581 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:00.097615 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 02:03:00.097668 (+ 53us) client_negotiation.cc:240] Negotiation successful
0809 02:03:00.097748 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":626,"mutex_wait_us":2}
I20260809 02:03:00.098325 9985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:00.093101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.193:37543 (local address 127.0.0.1:41786)
0809 02:03:00.093742 (+ 641us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:00.093759 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 02:03:00.093843 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:03:00.094190 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:03:00.094194 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:03:00.094210 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:03:00.094312 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:00.094318 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:00.095362 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:00.095365 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:00.096503 (+ 1138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:00.096512 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:00.097926 (+ 1414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:00.097929 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:00.098022 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:00.098056 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 02:03:00.098125 (+ 69us) client_negotiation.cc:240] Negotiation successful
0809 02:03:00.098195 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":553,"mutex_wait_us":7}
I20260809 02:03:00.098462 9999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:00.093422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43160 (local address 127.1.192.195:39329)
0809 02:03:00.093794 (+ 372us) server_negotiation.cc:206] Beginning negotiation
0809 02:03:00.093800 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:03:00.094118 (+ 318us) server_negotiation.cc:407] Connection header received
0809 02:03:00.094254 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:03:00.094257 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:03:00.094309 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:03:00.094708 (+ 399us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:03:00.094842 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:00.095552 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:00.096702 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:00.097302 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:00.098131 (+ 829us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:00.098211 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 02:03:00.098249 (+ 38us) server_negotiation.cc:299] Negotiation successful
0809 02:03:00.098323 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301,"thread_start_us":94,"threads_started":1}
I20260809 02:03:00.098932 10000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:00.093511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41786 (local address 127.1.192.193:37543)
0809 02:03:00.093779 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0809 02:03:00.093784 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:03:00.093802 (+ 18us) server_negotiation.cc:407] Connection header received
0809 02:03:00.093970 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:03:00.093980 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:03:00.094041 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:03:00.094177 (+ 136us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:03:00.094441 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:00.095204 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:00.097129 (+ 1925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:00.097783 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:00.098635 (+ 852us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:00.098703 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 02:03:00.098741 (+ 38us) server_negotiation.cc:299] Negotiation successful
0809 02:03:00.098820 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":77,"threads_started":1}
I20260809 02:03:00.099002 9822 tablet_service.cc:1467] Processing CreateTablet for tablet 4cad81a5af7a449ba011594226b8653e (DEFAULT_TABLE table=test-table [id=1ade27d141d446aebbca9592bb83542d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:03:00.099329 9822 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4cad81a5af7a449ba011594226b8653e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:03:00.099457 9523 tablet_service.cc:1467] Processing CreateTablet for tablet 4cad81a5af7a449ba011594226b8653e (DEFAULT_TABLE table=test-table [id=1ade27d141d446aebbca9592bb83542d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:03:00.099614 10002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:00.093612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.194:35937 (local address 127.0.0.1:50348)
0809 02:03:00.094649 (+ 1037us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:00.094663 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 02:03:00.094754 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:03:00.094997 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:03:00.095000 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:03:00.095018 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:03:00.095152 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:00.095159 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:00.096473 (+ 1314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:00.096476 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:00.097797 (+ 1321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:00.097808 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:00.099207 (+ 1399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:00.099214 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:00.099321 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:00.099345 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 02:03:00.099421 (+ 76us) client_negotiation.cc:240] Negotiation successful
0809 02:03:00.099489 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":967,"mutex_wait_us":23,"thread_start_us":130,"threads_started":1}
I20260809 02:03:00.099766 9523 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4cad81a5af7a449ba011594226b8653e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:03:00.100008 10003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:00.094334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50348 (local address 127.1.192.194:35937)
0809 02:03:00.094626 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0809 02:03:00.094631 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:03:00.094744 (+ 113us) server_negotiation.cc:407] Connection header received
0809 02:03:00.094836 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:03:00.094839 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:03:00.094888 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:03:00.094973 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:03:00.095277 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:00.096345 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:00.097913 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:00.098961 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:00.099069 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:00.099131 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 02:03:00.099777 (+ 646us) server_negotiation.cc:299] Negotiation successful
0809 02:03:00.099869 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":200,"thread_start_us":114,"threads_started":1}
I20260809 02:03:00.100464 9676 tablet_service.cc:1467] Processing CreateTablet for tablet 4cad81a5af7a449ba011594226b8653e (DEFAULT_TABLE table=test-table [id=1ade27d141d446aebbca9592bb83542d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:03:00.100744 9676 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4cad81a5af7a449ba011594226b8653e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:03:00.102146 10004 tablet_bootstrap.cc:492] T 4cad81a5af7a449ba011594226b8653e P cd2035d243e94eeea383bb2f79f51dc1: Bootstrap starting.
I20260809 02:03:00.102312 10005 tablet_bootstrap.cc:492] T 4cad81a5af7a449ba011594226b8653e P 2eb7b1762aec408499a7ed56b7923dfc: Bootstrap starting.
I20260809 02:03:00.103121 10004 tablet_bootstrap.cc:654] T 4cad81a5af7a449ba011594226b8653e P cd2035d243e94eeea383bb2f79f51dc1: Neither blocks nor log segments found. Creating new log.
I20260809 02:03:00.103221 10005 tablet_bootstrap.cc:654] T 4cad81a5af7a449ba011594226b8653e P 2eb7b1762aec408499a7ed56b7923dfc: Neither blocks nor log segments found. Creating new log.
I20260809 02:03:00.103458 10006 tablet_bootstrap.cc:492] T 4cad81a5af7a449ba011594226b8653e P 8030853bc1144362a198609d5c2a1a6d: Bootstrap starting.
I20260809 02:03:00.104497 10006 tablet_bootstrap.cc:654] T 4cad81a5af7a449ba011594226b8653e P 8030853bc1144362a198609d5c2a1a6d: Neither blocks nor log segments found. Creating new log.
I20260809 02:03:00.104825 10005 tablet_bootstrap.cc:492] T 4cad81a5af7a449ba011594226b8653e P 2eb7b1762aec408499a7ed56b7923dfc: No bootstrap required, opened a new log
I20260809 02:03:00.104837 10004 tablet_bootstrap.cc:492] T 4cad81a5af7a449ba011594226b8653e P cd2035d243e94eeea383bb2f79f51dc1: No bootstrap required, opened a new log
I20260809 02:03:00.104928 10004 ts_tablet_manager.cc:1397] T 4cad81a5af7a449ba011594226b8653e P cd2035d243e94eeea383bb2f79f51dc1: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 02:03:00.104935 10005 ts_tablet_manager.cc:1397] T 4cad81a5af7a449ba011594226b8653e P 2eb7b1762aec408499a7ed56b7923dfc: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 02:03:00.105302 10005 raft_consensus.cc:348] T 4cad81a5af7a449ba011594226b8653e P 2eb7b1762aec408499a7ed56b7923dfc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2eb7b1762aec408499a7ed56b7923dfc" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 37543 } } peers { permanent_uuid: "cd2035d243e94eeea383bb2f79f51dc1" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 39329 } } peers { permanent_uuid: "8030853bc1144362a198609d5c2a1a6d" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 35937 } }
I20260809 02:03:00.105319 10004 raft_consensus.cc:348] T 4cad81a5af7a449ba011594226b8653e P cd2035d243e94eeea383bb2f79f51dc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2eb7b1762aec408499a7ed56b7923dfc" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 37543 } } peers { permanent_uuid: "cd2035d243e94eeea383bb2f79f51dc1" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 39329 } } peers { permanent_uuid: "8030853bc1144362a198609d5c2a1a6d" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 35937 } }
I20260809 02:03:00.105422 10004 raft_consensus.cc:374] T 4cad81a5af7a449ba011594226b8653e P cd2035d243e94eeea383bb2f79f51dc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:03:00.105427 10005 raft_consensus.cc:374] T 4cad81a5af7a449ba011594226b8653e P 2eb7b1762aec408499a7ed56b7923dfc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:03:00.105443 10004 raft_consensus.cc:729] T 4cad81a5af7a449ba011594226b8653e P cd2035d243e94eeea383bb2f79f51dc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd2035d243e94eeea383bb2f79f51dc1, State: Initialized, Role: FOLLOWER
I20260809 02:03:00.105470 10005 raft_consensus.cc:729] T 4cad81a5af7a449ba011594226b8653e P 2eb7b1762aec408499a7ed56b7923dfc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2eb7b1762aec408499a7ed56b7923dfc, State: Initialized, Role: FOLLOWER
I20260809 02:03:00.105628 10004 consensus_queue.cc:260] T 4cad81a5af7a449ba011594226b8653e P cd2035d243e94eeea383bb2f79f51dc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2eb7b1762aec408499a7ed56b7923dfc" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 37543 } } peers { permanent_uuid: "cd2035d243e94eeea383bb2f79f51dc1" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 39329 } } peers { permanent_uuid: "8030853bc1144362a198609d5c2a1a6d" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 35937 } }
I20260809 02:03:00.105628 10005 consensus_queue.cc:260] T 4cad81a5af7a449ba011594226b8653e P 2eb7b1762aec408499a7ed56b7923dfc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2eb7b1762aec408499a7ed56b7923dfc" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 37543 } } peers { permanent_uuid: "cd2035d243e94eeea383bb2f79f51dc1" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 39329 } } peers { permanent_uuid: "8030853bc1144362a198609d5c2a1a6d" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 35937 } }
I20260809 02:03:00.105983 10005 ts_tablet_manager.cc:1428] T 4cad81a5af7a449ba011594226b8653e P 2eb7b1762aec408499a7ed56b7923dfc: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260809 02:03:00.105984 10004 ts_tablet_manager.cc:1428] T 4cad81a5af7a449ba011594226b8653e P cd2035d243e94eeea383bb2f79f51dc1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 02:03:00.106683 10013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:00.106280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:34543 (local address 127.1.192.195:38683)
0809 02:03:00.106512 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:00.106573 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:34543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":109,"threads_started":1}
I20260809 02:03:00.107805 10014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:00.107363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:34543 (local address 127.1.192.193:58655)
0809 02:03:00.107656 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:00.107716 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:34543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":100,"threads_started":1}
I20260809 02:03:00.108393 10013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:00.108159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:34543 (local address 127.1.192.195:49529)
0809 02:03:00.108278 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:00.108333 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:34543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260809 02:03:00.109014 10014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:00.108754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:34543 (local address 127.1.192.193:39513)
0809 02:03:00.108876 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:00.108939 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:34543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260809 02:03:00.109319 9901 heartbeater.cc:503] Master 127.1.192.252:44581 was elected leader, sending a full tablet report...
I20260809 02:03:00.109377 9601 heartbeater.cc:503] Master 127.1.192.252:44581 was elected leader, sending a full tablet report...
I20260809 02:03:00.110664 10006 tablet_bootstrap.cc:492] T 4cad81a5af7a449ba011594226b8653e P 8030853bc1144362a198609d5c2a1a6d: No bootstrap required, opened a new log
I20260809 02:03:00.110771 10006 ts_tablet_manager.cc:1397] T 4cad81a5af7a449ba011594226b8653e P 8030853bc1144362a198609d5c2a1a6d: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20260809 02:03:00.111212 10006 raft_consensus.cc:348] T 4cad81a5af7a449ba011594226b8653e P 8030853bc1144362a198609d5c2a1a6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2eb7b1762aec408499a7ed56b7923dfc" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 37543 } } peers { permanent_uuid: "cd2035d243e94eeea383bb2f79f51dc1" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 39329 } } peers { permanent_uuid: "8030853bc1144362a198609d5c2a1a6d" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 35937 } }
I20260809 02:03:00.111332 10006 raft_consensus.cc:374] T 4cad81a5af7a449ba011594226b8653e P 8030853bc1144362a198609d5c2a1a6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:03:00.111356 10006 raft_consensus.cc:729] T 4cad81a5af7a449ba011594226b8653e P 8030853bc1144362a198609d5c2a1a6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8030853bc1144362a198609d5c2a1a6d, State: Initialized, Role: FOLLOWER
I20260809 02:03:00.111464 10006 consensus_queue.cc:260] T 4cad81a5af7a449ba011594226b8653e P 8030853bc1144362a198609d5c2a1a6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2eb7b1762aec408499a7ed56b7923dfc" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 37543 } } peers { permanent_uuid: "cd2035d243e94eeea383bb2f79f51dc1" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 39329 } } peers { permanent_uuid: "8030853bc1144362a198609d5c2a1a6d" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 35937 } }
I20260809 02:03:00.111724 10006 ts_tablet_manager.cc:1428] T 4cad81a5af7a449ba011594226b8653e P 8030853bc1144362a198609d5c2a1a6d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 02:03:00.112512 10016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:00.112067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:34543 (local address 127.1.192.194:33847)
0809 02:03:00.112353 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:00.112410 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:34543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":114,"threads_started":1}
I20260809 02:03:00.113176 9747 heartbeater.cc:503] Master 127.1.192.252:44581 was elected leader, sending a full tablet report...
I20260809 02:03:00.113583 10016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:00.112978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:34543 (local address 127.1.192.194:51897)
0809 02:03:00.113082 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:00.113470 (+ 388us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:34543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260809 02:03:00.272925 9750 tablet.cc:2369] T 4cad81a5af7a449ba011594226b8653e P 8030853bc1144362a198609d5c2a1a6d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 02:03:00.300755 9604 tablet.cc:2369] T 4cad81a5af7a449ba011594226b8653e P 2eb7b1762aec408499a7ed56b7923dfc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 02:03:00.331575 10011 raft_consensus.cc:482] T 4cad81a5af7a449ba011594226b8653e P cd2035d243e94eeea383bb2f79f51dc1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:03:00.331754 10011 raft_consensus.cc:504] T 4cad81a5af7a449ba011594226b8653e P cd2035d243e94eeea383bb2f79f51dc1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2eb7b1762aec408499a7ed56b7923dfc" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 37543 } } peers { permanent_uuid: "cd2035d243e94eeea383bb2f79f51dc1" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 39329 } } peers { permanent_uuid: "8030853bc1144362a198609d5c2a1a6d" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 35937 } }
I20260809 02:03:00.333021 10011 leader_election.cc:288] T 4cad81a5af7a449ba011594226b8653e P cd2035d243e94eeea383bb2f79f51dc1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2eb7b1762aec408499a7ed56b7923dfc (127.1.192.193:37543), 8030853bc1144362a198609d5c2a1a6d (127.1.192.194:35937)
I20260809 02:03:00.338104 10000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:00.333401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.195:41129 (local address 127.1.192.193:37543)
0809 02:03:00.333555 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0809 02:03:00.333560 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:03:00.333577 (+ 17us) server_negotiation.cc:407] Connection header received
0809 02:03:00.333741 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:03:00.333745 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:03:00.333803 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:03:00.333862 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:03:00.334434 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:00.335343 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:00.336761 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:00.337330 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:00.337470 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:00.337558 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 02:03:00.337847 (+ 289us) server_negotiation.cc:299] Negotiation successful
0809 02:03:00.337942 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 02:03:00.338246 10003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:00.333778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.195:33269 (local address 127.1.192.194:35937)
0809 02:03:00.333937 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0809 02:03:00.333944 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 02:03:00.334102 (+ 158us) server_negotiation.cc:407] Connection header received
0809 02:03:00.334170 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:03:00.334173 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:03:00.334238 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:03:00.334329 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:03:00.334623 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:00.335462 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:00.337023 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:00.337652 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:00.337771 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:00.337819 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0809 02:03:00.338049 (+ 230us) server_negotiation.cc:299] Negotiation successful
0809 02:03:00.338125 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 02:03:00.338312 10017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:00.333564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.194:35937 (local address 127.1.192.195:33269)
0809 02:03:00.333959 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:00.333973 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 02:03:00.334089 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:03:00.334351 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:03:00.334354 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:03:00.334370 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:03:00.334489 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:00.334494 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:00.335579 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:00.335582 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:00.336873 (+ 1291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:00.336884 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:00.337805 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:00.337813 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:00.337917 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:00.337940 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 02:03:00.338021 (+ 81us) client_negotiation.cc:240] Negotiation successful
0809 02:03:00.338100 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":331,"mutex_wait_us":3,"thread_start_us":160,"threads_started":1}
I20260809 02:03:00.338572 10013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:00.333254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.193:37543 (local address 127.1.192.195:41129)
0809 02:03:00.333405 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:00.333424 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 02:03:00.333562 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:03:00.334069 (+ 507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:03:00.334072 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:03:00.334099 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:03:00.334279 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:00.334291 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:00.335509 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:00.335513 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:00.336564 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:00.336575 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:00.337471 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:00.337475 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:00.337653 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:00.337685 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 02:03:00.338342 (+ 657us) client_negotiation.cc:240] Negotiation successful
0809 02:03:00.338435 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58,"mutex_wait_us":62}
I20260809 02:03:00.338997 9697 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4cad81a5af7a449ba011594226b8653e" candidate_uuid: "cd2035d243e94eeea383bb2f79f51dc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8030853bc1144362a198609d5c2a1a6d" is_pre_election: true
I20260809 02:03:00.339190 9697 raft_consensus.cc:2454] T 4cad81a5af7a449ba011594226b8653e P 8030853bc1144362a198609d5c2a1a6d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd2035d243e94eeea383bb2f79f51dc1 in term 0.
I20260809 02:03:00.339150 9545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4cad81a5af7a449ba011594226b8653e" candidate_uuid: "cd2035d243e94eeea383bb2f79f51dc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2eb7b1762aec408499a7ed56b7923dfc" is_pre_election: true
I20260809 02:03:00.339378 9545 raft_consensus.cc:2454] T 4cad81a5af7a449ba011594226b8653e P 2eb7b1762aec408499a7ed56b7923dfc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd2035d243e94eeea383bb2f79f51dc1 in term 0.
I20260809 02:03:00.339644 9773 leader_election.cc:302] T 4cad81a5af7a449ba011594226b8653e P cd2035d243e94eeea383bb2f79f51dc1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8030853bc1144362a198609d5c2a1a6d, cd2035d243e94eeea383bb2f79f51dc1; no voters:
I20260809 02:03:00.339956 10011 raft_consensus.cc:2781] T 4cad81a5af7a449ba011594226b8653e P cd2035d243e94eeea383bb2f79f51dc1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:03:00.340026 10011 raft_consensus.cc:482] T 4cad81a5af7a449ba011594226b8653e P cd2035d243e94eeea383bb2f79f51dc1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:03:00.340067 10011 raft_consensus.cc:3037] T 4cad81a5af7a449ba011594226b8653e P cd2035d243e94eeea383bb2f79f51dc1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:03:00.341178 10011 raft_consensus.cc:504] T 4cad81a5af7a449ba011594226b8653e P cd2035d243e94eeea383bb2f79f51dc1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2eb7b1762aec408499a7ed56b7923dfc" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 37543 } } peers { permanent_uuid: "cd2035d243e94eeea383bb2f79f51dc1" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 39329 } } peers { permanent_uuid: "8030853bc1144362a198609d5c2a1a6d" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 35937 } }
I20260809 02:03:00.341643 10011 leader_election.cc:288] T 4cad81a5af7a449ba011594226b8653e P cd2035d243e94eeea383bb2f79f51dc1 [CANDIDATE]: Term 1 election: Requested vote from peers 2eb7b1762aec408499a7ed56b7923dfc (127.1.192.193:37543), 8030853bc1144362a198609d5c2a1a6d (127.1.192.194:35937)
I20260809 02:03:00.341990 9545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4cad81a5af7a449ba011594226b8653e" candidate_uuid: "cd2035d243e94eeea383bb2f79f51dc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2eb7b1762aec408499a7ed56b7923dfc"
I20260809 02:03:00.342077 9697 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4cad81a5af7a449ba011594226b8653e" candidate_uuid: "cd2035d243e94eeea383bb2f79f51dc1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8030853bc1144362a198609d5c2a1a6d"
I20260809 02:03:00.342132 9545 raft_consensus.cc:3037] T 4cad81a5af7a449ba011594226b8653e P 2eb7b1762aec408499a7ed56b7923dfc [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:03:00.342263 9697 raft_consensus.cc:3037] T 4cad81a5af7a449ba011594226b8653e P 8030853bc1144362a198609d5c2a1a6d [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:03:00.343283 9697 raft_consensus.cc:2454] T 4cad81a5af7a449ba011594226b8653e P 8030853bc1144362a198609d5c2a1a6d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd2035d243e94eeea383bb2f79f51dc1 in term 1.
I20260809 02:03:00.343290 9545 raft_consensus.cc:2454] T 4cad81a5af7a449ba011594226b8653e P 2eb7b1762aec408499a7ed56b7923dfc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd2035d243e94eeea383bb2f79f51dc1 in term 1.
I20260809 02:03:00.343606 9773 leader_election.cc:302] T 4cad81a5af7a449ba011594226b8653e P cd2035d243e94eeea383bb2f79f51dc1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8030853bc1144362a198609d5c2a1a6d, cd2035d243e94eeea383bb2f79f51dc1; no voters:
I20260809 02:03:00.343791 10011 raft_consensus.cc:2781] T 4cad81a5af7a449ba011594226b8653e P cd2035d243e94eeea383bb2f79f51dc1 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:03:00.344058 10011 raft_consensus.cc:686] T 4cad81a5af7a449ba011594226b8653e P cd2035d243e94eeea383bb2f79f51dc1 [term 1 LEADER]: Becoming Leader. State: Replica: cd2035d243e94eeea383bb2f79f51dc1, State: Running, Role: LEADER
I20260809 02:03:00.344377 10011 consensus_queue.cc:237] T 4cad81a5af7a449ba011594226b8653e P cd2035d243e94eeea383bb2f79f51dc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2eb7b1762aec408499a7ed56b7923dfc" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 37543 } } peers { permanent_uuid: "cd2035d243e94eeea383bb2f79f51dc1" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 39329 } } peers { permanent_uuid: "8030853bc1144362a198609d5c2a1a6d" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 35937 } }
I20260809 02:03:00.344887 10013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:00.344601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:34543 (local address 127.1.192.195:38697)
0809 02:03:00.344734 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:00.344811 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:34543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260809 02:03:00.347261 9397 catalog_manager.cc:5495] T 4cad81a5af7a449ba011594226b8653e P cd2035d243e94eeea383bb2f79f51dc1 reported cstate change: term changed from 0 to 1, leader changed from <none> to cd2035d243e94eeea383bb2f79f51dc1 (127.1.192.195). New cstate: current_term: 1 leader_uuid: "cd2035d243e94eeea383bb2f79f51dc1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2eb7b1762aec408499a7ed56b7923dfc" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 37543 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd2035d243e94eeea383bb2f79f51dc1" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 39329 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8030853bc1144362a198609d5c2a1a6d" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 35937 } health_report { overall_health: UNKNOWN } } }
I20260809 02:03:00.393617 9999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:00.389964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43168 (local address 127.1.192.195:39329)
0809 02:03:00.390091 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0809 02:03:00.390101 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 02:03:00.390113 (+ 12us) server_negotiation.cc:407] Connection header received
0809 02:03:00.390195 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:03:00.390197 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:03:00.390249 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:03:00.390375 (+ 126us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:03:00.390650 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:00.391179 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:00.392050 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:00.392283 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:00.392396 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:00.392698 (+ 302us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:03:00.393166 (+ 468us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:03:00.393304 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0809 02:03:00.393369 (+ 65us) server_negotiation.cc:299] Negotiation successful
0809 02:03:00.393431 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 02:03:00.396914 9697 raft_consensus.cc:1261] T 4cad81a5af7a449ba011594226b8653e P 8030853bc1144362a198609d5c2a1a6d [term 1 FOLLOWER]: Refusing update from remote peer cd2035d243e94eeea383bb2f79f51dc1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:03:00.397130 9545 raft_consensus.cc:1261] T 4cad81a5af7a449ba011594226b8653e P 2eb7b1762aec408499a7ed56b7923dfc [term 1 FOLLOWER]: Refusing update from remote peer cd2035d243e94eeea383bb2f79f51dc1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:03:00.397730 10011 consensus_queue.cc:1035] T 4cad81a5af7a449ba011594226b8653e P cd2035d243e94eeea383bb2f79f51dc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8030853bc1144362a198609d5c2a1a6d" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 35937 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:03:00.397980 10019 consensus_queue.cc:1035] T 4cad81a5af7a449ba011594226b8653e P cd2035d243e94eeea383bb2f79f51dc1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2eb7b1762aec408499a7ed56b7923dfc" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 37543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:03:00.399865 10016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:00.399538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:34543 (local address 127.1.192.194:42551)
0809 02:03:00.399715 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:00.399783 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:34543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
I20260809 02:03:00.402333 10014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:00.402061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:34543 (local address 127.1.192.193:34445)
0809 02:03:00.402164 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:00.402243 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:34543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260809 02:03:00.404295 10013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:00.403973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:34543 (local address 127.1.192.195:56153)
0809 02:03:00.404150 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:00.404213 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:34543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
I20260809 02:03:00.405066 10016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:00.404670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:34543 (local address 127.1.192.194:46451)
0809 02:03:00.404899 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:00.404982 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:34543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158}
I20260809 02:03:00.405340 10013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:00.405060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:34543 (local address 127.1.192.195:40019)
0809 02:03:00.405218 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:00.405270 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:34543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260809 02:03:00.405629 10024 mvcc.cc:205] Tried to move back new op lower bound from 7316443055699644416 to 7316443055493795840. Current Snapshot: MvccSnapshot[applied={T|T < 7316443055699644416}]
I20260809 02:03:00.405804 10025 mvcc.cc:205] Tried to move back new op lower bound from 7316443055699644416 to 7316443055493795840. Current Snapshot: MvccSnapshot[applied={T|T < 7316443055699644416 or (T in {7316443055699644416})}]
I20260809 02:03:00.406136 10014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:00.404843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:34543 (local address 127.1.192.193:54421)
0809 02:03:00.405998 (+ 1155us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:00.406058 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:34543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1088}
I20260809 02:03:00.408329 10022 mvcc.cc:205] Tried to move back new op lower bound from 7316443055699644416 to 7316443055493795840. Current Snapshot: MvccSnapshot[applied={T|T < 7316443055699644416 or (T in {7316443055699644416})}]
I20260809 02:03:00.414574 9398 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32880:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:03:00.414815 9398 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32880:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:03:00.419480 9398 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d909896773e04b93bbb1e1e7d4c11264: Sending DeleteTablet for 3 replicas of tablet 4cad81a5af7a449ba011594226b8653e
I20260809 02:03:00.420331 9822 tablet_service.cc:1514] Processing DeleteTablet for tablet 4cad81a5af7a449ba011594226b8653e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:03:00 UTC) from {username='oryx', principal='oryx/127.1.192.252@KRBTEST.COM'} at 127.0.0.1:43160
I20260809 02:03:00.420789 10035 tablet_replica.cc:330] T 4cad81a5af7a449ba011594226b8653e P cd2035d243e94eeea383bb2f79f51dc1: stopping tablet replica
I20260809 02:03:00.420761 9523 tablet_service.cc:1514] Processing DeleteTablet for tablet 4cad81a5af7a449ba011594226b8653e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:03:00 UTC) from {username='oryx', principal='oryx/127.1.192.252@KRBTEST.COM'} at 127.0.0.1:41786
I20260809 02:03:00.420928 10035 raft_consensus.cc:2229] T 4cad81a5af7a449ba011594226b8653e P cd2035d243e94eeea383bb2f79f51dc1 [term 1 LEADER]: Raft consensus shutting down.
I20260809 02:03:00.421177 10036 tablet_replica.cc:330] T 4cad81a5af7a449ba011594226b8653e P 2eb7b1762aec408499a7ed56b7923dfc: stopping tablet replica
I20260809 02:03:00.421315 10035 raft_consensus.cc:2258] T 4cad81a5af7a449ba011594226b8653e P cd2035d243e94eeea383bb2f79f51dc1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:03:00.421380 10036 raft_consensus.cc:2229] T 4cad81a5af7a449ba011594226b8653e P 2eb7b1762aec408499a7ed56b7923dfc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:03:00.421577 9676 tablet_service.cc:1514] Processing DeleteTablet for tablet 4cad81a5af7a449ba011594226b8653e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:03:00 UTC) from {username='oryx', principal='oryx/127.1.192.252@KRBTEST.COM'} at 127.0.0.1:50348
I20260809 02:03:00.421723 10036 raft_consensus.cc:2258] T 4cad81a5af7a449ba011594226b8653e P 2eb7b1762aec408499a7ed56b7923dfc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:03:00.422143 10035 ts_tablet_manager.cc:1905] T 4cad81a5af7a449ba011594226b8653e P cd2035d243e94eeea383bb2f79f51dc1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:03:00.422683 10036 ts_tablet_manager.cc:1905] T 4cad81a5af7a449ba011594226b8653e P 2eb7b1762aec408499a7ed56b7923dfc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:03:00.423185 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 9352
I20260809 02:03:00.425578 10035 ts_tablet_manager.cc:1918] T 4cad81a5af7a449ba011594226b8653e P cd2035d243e94eeea383bb2f79f51dc1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:03:00.425688 10035 log.cc:1196] T 4cad81a5af7a449ba011594226b8653e P cd2035d243e94eeea383bb2f79f51dc1: Deleting WAL directory at /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/ts-2/wal/wals/4cad81a5af7a449ba011594226b8653e
I20260809 02:03:00.426009 10035 ts_tablet_manager.cc:1939] T 4cad81a5af7a449ba011594226b8653e P cd2035d243e94eeea383bb2f79f51dc1: Deleting consensus metadata
I20260809 02:03:00.426707 9382 catalog_manager.cc:4841] TS cd2035d243e94eeea383bb2f79f51dc1 (127.1.192.195:39329): tablet 4cad81a5af7a449ba011594226b8653e (table test-table [id=1ade27d141d446aebbca9592bb83542d]) successfully deleted
I20260809 02:03:00.428361 10037 tablet_replica.cc:330] T 4cad81a5af7a449ba011594226b8653e P 8030853bc1144362a198609d5c2a1a6d: stopping tablet replica
I20260809 02:03:00.428534 10037 raft_consensus.cc:2229] T 4cad81a5af7a449ba011594226b8653e P 8030853bc1144362a198609d5c2a1a6d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:03:00.428704 10037 raft_consensus.cc:2258] T 4cad81a5af7a449ba011594226b8653e P 8030853bc1144362a198609d5c2a1a6d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:03:00.429468 10037 ts_tablet_manager.cc:1905] T 4cad81a5af7a449ba011594226b8653e P 8030853bc1144362a198609d5c2a1a6d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:03:00.431514 10037 ts_tablet_manager.cc:1918] T 4cad81a5af7a449ba011594226b8653e P 8030853bc1144362a198609d5c2a1a6d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:03:00.431622 10037 log.cc:1196] T 4cad81a5af7a449ba011594226b8653e P 8030853bc1144362a198609d5c2a1a6d: Deleting WAL directory at /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786240941968412-1795-0/minicluster-data/ts-1/wal/wals/4cad81a5af7a449ba011594226b8653e
I20260809 02:03:00.431877 10037 ts_tablet_manager.cc:1939] T 4cad81a5af7a449ba011594226b8653e P 8030853bc1144362a198609d5c2a1a6d: Deleting consensus metadata
I20260809 02:03:00.432646 9379 catalog_manager.cc:4841] TS 8030853bc1144362a198609d5c2a1a6d (127.1.192.194:35937): tablet 4cad81a5af7a449ba011594226b8653e (table test-table [id=1ade27d141d446aebbca9592bb83542d]) successfully deleted
I20260809 02:03:00.436506 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 9609
I20260809 02:03:00.436530 10002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:00.436233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.193:37543 (local address 127.0.0.1:41788)
0809 02:03:00.436378 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:00.436440 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.193:37543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260809 02:03:00.439731 9379 catalog_manager.cc:4584] TS 2eb7b1762aec408499a7ed56b7923dfc (127.1.192.193:37543): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4cad81a5af7a449ba011594226b8653e: Network error: Client connection negotiation failed: client connection to 127.1.192.193:37543: connect: Connection refused (error 111)
I20260809 02:03:00.444622 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 9755
I20260809 02:03:00.453872 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 9041
I20260809 02:03:00.462160 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 9159
2026-08-09T02:03:00Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (5762 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786240941968412-1795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:03:00 dist-test-slave-ss4n krb5kdc[10040](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:03:00 dist-test-slave-ss4n krb5kdc[10040](info): set up 2 sockets
Aug 09 02:03:00 dist-test-slave-ss4n krb5kdc[10040](info): commencing operation
krb5kdc: starting...
W20260809 02:03:01.564081 1795 mini_kdc.cc:121] Time spent starting KDC: real 1.074s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786240941968412-1795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786240941968412-1795-0/krb5kdc/test-user.keytab.
Aug 09 02:03:01 dist-test-slave-ss4n krb5kdc[10040](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240981, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:03:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:03:01Z Disabled control of system clock
WARNING: no policy specified for oryx/127.1.192.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.192.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.192.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786240941968412-1795-0/krb5kdc/oryx_127.1.192.254.keytab.
Entry for principal oryx/127.1.192.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786240941968412-1795-0/krb5kdc/oryx_127.1.192.254.keytab.
I20260809 02:03:01.705196 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786240941968412-1795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786240941968412-1795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786240941968412-1795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.192.254:33057
--webserver_interface=127.1.192.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786240941968412-1795-0/krb5kdc/oryx_127.1.192.254.keytab
--principal=oryx/127.1.192.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:40857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.192.254:33057
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:03:01.819362 10056 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:03:01.819715 10056 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:03:01.819801 10056 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:03:01.824822 10056 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:03:01.824954 10056 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 02:03:01.824990 10056 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:03:01.825035 10056 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:03:01.825075 10056 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:03:01.825114 10056 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:03:01.831509 10056 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:40857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786240941968412-1795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786240941968412-1795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.192.254:33057
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786240941968412-1795-0/krb5kdc/oryx_127.1.192.254.keytab
--principal=oryx/127.1.192.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.192.254:33057
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.192.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10056
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786240941968412-1795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:03:01.832732 10056 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:03:01.833853 10056 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:03:01.842976 10062 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:03:01.842976 10063 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:03:01.843047 10065 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:03:01.843253 10056 server_base.cc:1034] running on GCE node
I20260809 02:03:01.843948 10056 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:03:01.845150 10056 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:03:01.846344 10056 hybrid_clock.cc:648] HybridClock initialized: now 1786240981846304 us; error 57 us; skew 500 ppm
Aug 09 02:03:01 dist-test-slave-ss4n krb5kdc[10040](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240981, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.192.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:03:01.849269 10056 init.cc:377] Logged in from keytab as oryx/127.1.192.254@KRBTEST.COM (short username oryx)
I20260809 02:03:01.850257 10056 webserver.cc:460] Webserver started at http://127.1.192.254:35389/ using document root <none> and password file <none>
I20260809 02:03:01.850781 10056 fs_manager.cc:359] Metadata directory not provided
I20260809 02:03:01.850841 10056 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:03:01.851011 10056 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:03:01.853062 10056 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786240941968412-1795-0/minicluster-data/master-0/data/instance:
uuid: "18e8730c5b9e4ae28cf2094606dadc7b"
format_stamp: "Formatted at 2026-08-09 02:03:01 on dist-test-slave-ss4n"
server_key: "0a65830edf90d3cba83c5c3167230cc0"
server_key_iv: "22d96de507929b66dfee4fea64f24c4d"
server_key_version: "encryptionkey@0"
I20260809 02:03:01.853673 10056 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786240941968412-1795-0/minicluster-data/master-0/wal/instance:
uuid: "18e8730c5b9e4ae28cf2094606dadc7b"
format_stamp: "Formatted at 2026-08-09 02:03:01 on dist-test-slave-ss4n"
server_key: "0a65830edf90d3cba83c5c3167230cc0"
server_key_iv: "22d96de507929b66dfee4fea64f24c4d"
server_key_version: "encryptionkey@0"
I20260809 02:03:01.858589 10056 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260809 02:03:01.861131 10072 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:03:01.862672 10056 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 02:03:01.862785 10056 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786240941968412-1795-0/minicluster-data/master-0/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786240941968412-1795-0/minicluster-data/master-0/wal
uuid: "18e8730c5b9e4ae28cf2094606dadc7b"
format_stamp: "Formatted at 2026-08-09 02:03:01 on dist-test-slave-ss4n"
server_key: "0a65830edf90d3cba83c5c3167230cc0"
server_key_iv: "22d96de507929b66dfee4fea64f24c4d"
server_key_version: "encryptionkey@0"
I20260809 02:03:01.862883 10056 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786240941968412-1795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786240941968412-1795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786240941968412-1795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:03:01.878046 10056 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:03:01.880959 10056 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:03:01.881116 10056 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:03:01.889741 10056 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.254:33057
I20260809 02:03:01.889815 10137 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.254:33057 every 8 connection(s)
I20260809 02:03:01.890763 10056 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
I20260809 02:03:01.893045 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 10056
I20260809 02:03:01.893190 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786240941968412-1795-0/minicluster-data/master-0/wal/instance
I20260809 02:03:01.893612 1795 external_mini_cluster.cc:1428] Setting key 204fa924f5baf9e18216761b4d0926ea
I20260809 02:03:01.894223 10138 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:03:01.900743 10138 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 18e8730c5b9e4ae28cf2094606dadc7b: Bootstrap starting.
Aug 09 02:03:01 dist-test-slave-ss4n krb5kdc[10040](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240981, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM
I20260809 02:03:01.903399 10138 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 18e8730c5b9e4ae28cf2094606dadc7b: Neither blocks nor log segments found. Creating new log.
I20260809 02:03:01.904556 10138 log.cc:824] T 00000000000000000000000000000000 P 18e8730c5b9e4ae28cf2094606dadc7b: Log is configured to *not* fsync() on all Append() calls
I20260809 02:03:01.906749 10138 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 18e8730c5b9e4ae28cf2094606dadc7b: No bootstrap required, opened a new log
I20260809 02:03:01.908211 10142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:01.895147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51362 (local address 127.1.192.254:33057)
0809 02:03:01.895626 (+ 479us) server_negotiation.cc:206] Beginning negotiation
0809 02:03:01.895648 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0809 02:03:01.895685 (+ 37us) server_negotiation.cc:407] Connection header received
0809 02:03:01.896345 (+ 660us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:03:01.896360 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:03:01.896689 (+ 329us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:03:01.897026 (+ 337us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:03:01.898017 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:01.899044 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:01.899945 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:01.900431 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:01.900611 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:01.903209 (+ 2598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:03:01.903214 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:03:01.903228 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:03:01.903259 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:03:01.905127 (+ 1868us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:03:01.905628 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:03:01.905634 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:03:01.905639 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:03:01.905759 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:03:01.906068 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:03:01.906072 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:03:01.906073 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:03:01.906471 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:03:01.906697 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0809 02:03:01.906980 (+ 283us) server_negotiation.cc:299] Negotiation successful
0809 02:03:01.907268 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":163,"threads_started":1}
I20260809 02:03:01.912056 10138 raft_consensus.cc:348] T 00000000000000000000000000000000 P 18e8730c5b9e4ae28cf2094606dadc7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18e8730c5b9e4ae28cf2094606dadc7b" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 33057 } }
I20260809 02:03:01.912398 10138 raft_consensus.cc:374] T 00000000000000000000000000000000 P 18e8730c5b9e4ae28cf2094606dadc7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:03:01.912496 10138 raft_consensus.cc:729] T 00000000000000000000000000000000 P 18e8730c5b9e4ae28cf2094606dadc7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18e8730c5b9e4ae28cf2094606dadc7b, State: Initialized, Role: FOLLOWER
I20260809 02:03:01.913061 10138 consensus_queue.cc:260] T 00000000000000000000000000000000 P 18e8730c5b9e4ae28cf2094606dadc7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18e8730c5b9e4ae28cf2094606dadc7b" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 33057 } }
I20260809 02:03:01.913237 10138 raft_consensus.cc:388] T 00000000000000000000000000000000 P 18e8730c5b9e4ae28cf2094606dadc7b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:03:01.913338 10138 raft_consensus.cc:482] T 00000000000000000000000000000000 P 18e8730c5b9e4ae28cf2094606dadc7b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:03:01.913461 10138 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 18e8730c5b9e4ae28cf2094606dadc7b [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:03:01.914824 10138 raft_consensus.cc:504] T 00000000000000000000000000000000 P 18e8730c5b9e4ae28cf2094606dadc7b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18e8730c5b9e4ae28cf2094606dadc7b" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 33057 } }
I20260809 02:03:01.915227 10138 leader_election.cc:302] T 00000000000000000000000000000000 P 18e8730c5b9e4ae28cf2094606dadc7b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 18e8730c5b9e4ae28cf2094606dadc7b; no voters:
I20260809 02:03:01.915628 10138 leader_election.cc:288] T 00000000000000000000000000000000 P 18e8730c5b9e4ae28cf2094606dadc7b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:03:01.915779 10145 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 18e8730c5b9e4ae28cf2094606dadc7b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:03:01.916043 10145 raft_consensus.cc:686] T 00000000000000000000000000000000 P 18e8730c5b9e4ae28cf2094606dadc7b [term 1 LEADER]: Becoming Leader. State: Replica: 18e8730c5b9e4ae28cf2094606dadc7b, State: Running, Role: LEADER
I20260809 02:03:01.916419 10145 consensus_queue.cc:237] T 00000000000000000000000000000000 P 18e8730c5b9e4ae28cf2094606dadc7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18e8730c5b9e4ae28cf2094606dadc7b" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 33057 } }
I20260809 02:03:01.917160 10138 sys_catalog.cc:564] T 00000000000000000000000000000000 P 18e8730c5b9e4ae28cf2094606dadc7b [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:03:01.918633 10147 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18e8730c5b9e4ae28cf2094606dadc7b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "18e8730c5b9e4ae28cf2094606dadc7b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18e8730c5b9e4ae28cf2094606dadc7b" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 33057 } } }
I20260809 02:03:01.918787 10147 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18e8730c5b9e4ae28cf2094606dadc7b [sys.catalog]: This master's current role is: LEADER
I20260809 02:03:01.919199 10154 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:03:01.919322 10146 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18e8730c5b9e4ae28cf2094606dadc7b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 18e8730c5b9e4ae28cf2094606dadc7b. Latest consensus state: current_term: 1 leader_uuid: "18e8730c5b9e4ae28cf2094606dadc7b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18e8730c5b9e4ae28cf2094606dadc7b" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 33057 } } }
I20260809 02:03:01.919538 10146 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18e8730c5b9e4ae28cf2094606dadc7b [sys.catalog]: This master's current role is: LEADER
I20260809 02:03:01.923092 10154 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:03:01.934084 10154 catalog_manager.cc:1347] Generated new cluster ID: b888012ef0d7494ab3ec1efdad3dac3c
I20260809 02:03:01.934172 10154 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:03:01.941308 10154 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:03:01.942725 10154 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:03:01.965320 10154 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 18e8730c5b9e4ae28cf2094606dadc7b: Generated new TSK 0
I20260809 02:03:01.966267 10154 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.1.192.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.192.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.192.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786240941968412-1795-0/krb5kdc/oryx_127.1.192.193.keytab.
Entry for principal oryx/127.1.192.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786240941968412-1795-0/krb5kdc/oryx_127.1.192.193.keytab.
I20260809 02:03:02.005712 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786240941968412-1795-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786240941968412-1795-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786240941968412-1795-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786240941968412-1795-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.192.193:0
--local_ip_for_outbound_sockets=127.1.192.193
--webserver_interface=127.1.192.193
--webserver_port=0
--tserver_master_addrs=127.1.192.254:33057
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786240941968412-1795-0/krb5kdc/oryx_127.1.192.193.keytab
--principal=oryx/127.1.192.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:40857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:03:02.142272 10166 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:03:02.142549 10166 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:03:02.142598 10166 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:03:02.146761 10166 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:03:02.146828 10166 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:03:02.146869 10166 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 02:03:02.146936 10166 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.192.193
I20260809 02:03:02.152302 10166 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:40857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786240941968412-1795-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786240941968412-1795-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786240941968412-1795-0/krb5kdc/oryx_127.1.192.193.keytab
--principal=oryx/127.1.192.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.192.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786240941968412-1795-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.192.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.192.254:33057
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10166
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.192.193
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786240941968412-1795-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:03:02.153411 10166 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:03:02.154470 10166 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:03:02.164631 10175 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:03:02.164624 10173 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:03:02.164799 10166 server_base.cc:1034] running on GCE node
W20260809 02:03:02.164624 10172 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:03:02.165475 10166 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:03:02.166263 10166 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:03:02.167431 10166 hybrid_clock.cc:648] HybridClock initialized: now 1786240982167399 us; error 48 us; skew 500 ppm
Aug 09 02:03:02 dist-test-slave-ss4n krb5kdc[10040](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240982, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.192.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:03:02.170372 10166 init.cc:377] Logged in from keytab as oryx/127.1.192.193@KRBTEST.COM (short username oryx)
I20260809 02:03:02.171303 10166 webserver.cc:460] Webserver started at http://127.1.192.193:36429/ using document root <none> and password file <none>
I20260809 02:03:02.171880 10166 fs_manager.cc:359] Metadata directory not provided
I20260809 02:03:02.171940 10166 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:03:02.172204 10166 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:03:02.174517 10166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786240941968412-1795-0/minicluster-data/ts-0/data/instance:
uuid: "561e9aefe338421caf224b517f925355"
format_stamp: "Formatted at 2026-08-09 02:03:02 on dist-test-slave-ss4n"
server_key: "e88f859693d07db74f9ab8dbf4b12d3f"
server_key_iv: "e8052d75f7f09f17b846905ecc3f6861"
server_key_version: "encryptionkey@0"
I20260809 02:03:02.175087 10166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786240941968412-1795-0/minicluster-data/ts-0/wal/instance:
uuid: "561e9aefe338421caf224b517f925355"
format_stamp: "Formatted at 2026-08-09 02:03:02 on dist-test-slave-ss4n"
server_key: "e88f859693d07db74f9ab8dbf4b12d3f"
server_key_iv: "e8052d75f7f09f17b846905ecc3f6861"
server_key_version: "encryptionkey@0"
I20260809 02:03:02.178918 10166 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 02:03:02.181277 10183 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:03:02.182546 10166 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 02:03:02.182701 10166 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786240941968412-1795-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786240941968412-1795-0/minicluster-data/ts-0/wal
uuid: "561e9aefe338421caf224b517f925355"
format_stamp: "Formatted at 2026-08-09 02:03:02 on dist-test-slave-ss4n"
server_key: "e88f859693d07db74f9ab8dbf4b12d3f"
server_key_iv: "e8052d75f7f09f17b846905ecc3f6861"
server_key_version: "encryptionkey@0"
I20260809 02:03:02.182821 10166 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786240941968412-1795-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786240941968412-1795-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786240941968412-1795-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:03:02.199710 10166 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:03:02.202919 10166 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:03:02.203142 10166 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:03:02.204700 10166 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:03:02.204773 10166 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:03:02.204850 10166 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:03:02.204895 10166 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 02:03:02 dist-test-slave-ss4n krb5kdc[10040](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240982, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.192.193@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM
I20260809 02:03:02.216205 10166 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.193:35291
I20260809 02:03:02.216383 10311 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.193:35291 every 8 connection(s)
I20260809 02:03:02.217337 10166 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786240941968412-1795-0/minicluster-data/ts-0/data/info.pb
I20260809 02:03:02.219115 10142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:02.205614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.193:54927 (local address 127.1.192.254:33057)
0809 02:03:02.205804 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 02:03:02.205809 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:03:02.206662 (+ 853us) server_negotiation.cc:407] Connection header received
0809 02:03:02.207754 (+ 1092us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:03:02.207759 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:03:02.207827 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:03:02.207914 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:03:02.209454 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:02.210039 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:02.211197 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:02.211505 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:02.211609 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:02.214985 (+ 3376us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:03:02.214989 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:03:02.214992 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:03:02.215024 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:03:02.216788 (+ 1764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:03:02.217367 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:03:02.217371 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:03:02.217373 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:03:02.217506 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:03:02.217845 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:03:02.217848 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:03:02.217851 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:03:02.218127 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:03:02.218239 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 02:03:02.218801 (+ 562us) server_negotiation.cc:299] Negotiation successful
0809 02:03:02.218940 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260809 02:03:02.219944 10196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:02.206033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:33057 (local address 127.1.192.193:54927)
0809 02:03:02.206483 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:02.206535 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 02:03:02.207486 (+ 951us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:03:02.208303 (+ 817us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:03:02.208312 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:03:02.208777 (+ 465us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:03:02.209302 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:02.209316 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:02.210182 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:02.210187 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:02.211062 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:02.211071 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:02.211713 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:02.211720 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:02.211977 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:02.212690 (+ 713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:03:02.212717 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:03:02.214807 (+ 2090us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:03:02.216906 (+ 2099us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:03:02.216912 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:03:02.216926 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:03:02.217258 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:03:02.217589 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:03:02.217592 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:03:02.217594 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:03:02.217743 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:03:02.218249 (+ 506us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:03:02.218257 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:03:02.218544 (+ 287us) client_negotiation.cc:769] Sending connection context
0809 02:03:02.218743 (+ 199us) client_negotiation.cc:240] Negotiation successful
0809 02:03:02.219067 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":132,"threads_started":1}
I20260809 02:03:02.221274 10312 heartbeater.cc:347] Connected to a master server at 127.1.192.254:33057
I20260809 02:03:02.221451 10312 heartbeater.cc:464] Registering TS with master...
I20260809 02:03:02.222149 10312 heartbeater.cc:511] Master 127.1.192.254:33057 requested a full tablet report, sending...
I20260809 02:03:02.223829 10088 ts_manager.cc:194] Registered new tserver with Master: 561e9aefe338421caf224b517f925355 (127.1.192.193:35291)
I20260809 02:03:02.224900 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 10166
I20260809 02:03:02.225041 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786240941968412-1795-0/minicluster-data/ts-0/wal/instance
I20260809 02:03:02.225342 1795 external_mini_cluster.cc:1428] Setting key c2a5afbcb9fa579d65b092f1de9b0715
I20260809 02:03:02.225317 10088 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.192.193@KRBTEST.COM'} at 127.1.192.193:54927
I20260809 02:03:02.231560 10319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:02.223374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:33057 (local address 127.1.192.193:35733)
0809 02:03:02.223638 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:02.223654 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 02:03:02.223751 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:03:02.223998 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:03:02.224001 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:03:02.224192 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:03:02.224301 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:02.224307 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:02.225362 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:02.225366 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:02.226203 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:02.226213 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:02.226803 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:02.226806 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:02.226907 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:02.227458 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:03:02.227473 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:03:02.227915 (+ 442us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:03:02.229861 (+ 1946us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:03:02.229865 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:03:02.229869 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:03:02.230143 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:03:02.230482 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:03:02.230487 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:03:02.230489 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:03:02.230578 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:03:02.231108 (+ 530us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:03:02.231110 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:03:02.231212 (+ 102us) client_negotiation.cc:769] Sending connection context
0809 02:03:02.231299 (+ 87us) client_negotiation.cc:240] Negotiation successful
0809 02:03:02.231410 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":111,"threads_started":1}
I20260809 02:03:02.231776 10142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:02.223464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.193:35733 (local address 127.1.192.254:33057)
0809 02:03:02.223570 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0809 02:03:02.223574 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:03:02.223731 (+ 157us) server_negotiation.cc:407] Connection header received
0809 02:03:02.223826 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:03:02.223829 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:03:02.223876 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:03:02.223963 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:03:02.224413 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:02.225202 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:02.226327 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:02.226681 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:02.226799 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:02.228079 (+ 1280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:03:02.228086 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:03:02.228092 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:03:02.228125 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:03:02.229728 (+ 1603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:03:02.230291 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:03:02.230294 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:03:02.230295 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:03:02.230377 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:03:02.230701 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:03:02.230708 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:03:02.230712 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:03:02.230970 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:03:02.231113 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0809 02:03:02.231387 (+ 274us) server_negotiation.cc:299] Negotiation successful
0809 02:03:02.231538 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":42}
WARNING: no policy specified for oryx/127.1.192.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.192.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.192.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786240941968412-1795-0/krb5kdc/oryx_127.1.192.194.keytab.
Entry for principal oryx/127.1.192.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786240941968412-1795-0/krb5kdc/oryx_127.1.192.194.keytab.
I20260809 02:03:02.257903 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786240941968412-1795-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786240941968412-1795-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786240941968412-1795-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786240941968412-1795-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.192.194:0
--local_ip_for_outbound_sockets=127.1.192.194
--webserver_interface=127.1.192.194
--webserver_port=0
--tserver_master_addrs=127.1.192.254:33057
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786240941968412-1795-0/krb5kdc/oryx_127.1.192.194.keytab
--principal=oryx/127.1.192.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:40857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:03:02.377885 10322 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:03:02.378196 10322 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:03:02.378242 10322 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:03:02.382333 10322 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:03:02.382397 10322 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:03:02.382437 10322 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 02:03:02.382470 10322 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.192.194
I20260809 02:03:02.387961 10322 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:40857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786240941968412-1795-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786240941968412-1795-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786240941968412-1795-0/krb5kdc/oryx_127.1.192.194.keytab
--principal=oryx/127.1.192.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.192.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786240941968412-1795-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.192.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.192.254:33057
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10322
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.192.194
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786240941968412-1795-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:03:02.389091 10322 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:03:02.390080 10322 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:03:02.399632 10329 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:03:02.399652 10332 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:03:02.399744 10322 server_base.cc:1034] running on GCE node
W20260809 02:03:02.399624 10330 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:03:02.400298 10322 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:03:02.400949 10322 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:03:02.402139 10322 hybrid_clock.cc:648] HybridClock initialized: now 1786240982402122 us; error 29 us; skew 500 ppm
Aug 09 02:03:02 dist-test-slave-ss4n krb5kdc[10040](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240982, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.192.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:03:02.405139 10322 init.cc:377] Logged in from keytab as oryx/127.1.192.194@KRBTEST.COM (short username oryx)
I20260809 02:03:02.406234 10322 webserver.cc:460] Webserver started at http://127.1.192.194:44099/ using document root <none> and password file <none>
I20260809 02:03:02.406840 10322 fs_manager.cc:359] Metadata directory not provided
I20260809 02:03:02.406944 10322 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:03:02.407230 10322 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:03:02.409597 10322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786240941968412-1795-0/minicluster-data/ts-1/data/instance:
uuid: "8fd0ea46d91d44d685754406c9f842cf"
format_stamp: "Formatted at 2026-08-09 02:03:02 on dist-test-slave-ss4n"
server_key: "fc3d51dc928df70c59a09213f674cfa3"
server_key_iv: "b9172e699c72d142f2cc55b62175e821"
server_key_version: "encryptionkey@0"
I20260809 02:03:02.410279 10322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786240941968412-1795-0/minicluster-data/ts-1/wal/instance:
uuid: "8fd0ea46d91d44d685754406c9f842cf"
format_stamp: "Formatted at 2026-08-09 02:03:02 on dist-test-slave-ss4n"
server_key: "fc3d51dc928df70c59a09213f674cfa3"
server_key_iv: "b9172e699c72d142f2cc55b62175e821"
server_key_version: "encryptionkey@0"
I20260809 02:03:02.414264 10322 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260809 02:03:02.416886 10340 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:03:02.418514 10322 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 02:03:02.418668 10322 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786240941968412-1795-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786240941968412-1795-0/minicluster-data/ts-1/wal
uuid: "8fd0ea46d91d44d685754406c9f842cf"
format_stamp: "Formatted at 2026-08-09 02:03:02 on dist-test-slave-ss4n"
server_key: "fc3d51dc928df70c59a09213f674cfa3"
server_key_iv: "b9172e699c72d142f2cc55b62175e821"
server_key_version: "encryptionkey@0"
I20260809 02:03:02.418792 10322 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786240941968412-1795-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786240941968412-1795-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786240941968412-1795-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:03:02.431881 10322 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:03:02.434830 10322 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:03:02.435051 10322 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:03:02.436529 10322 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:03:02.436601 10322 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:03:02.436677 10322 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:03:02.436733 10322 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 02:03:02 dist-test-slave-ss4n krb5kdc[10040](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240982, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.192.194@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM
I20260809 02:03:02.448232 10322 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.194:35667
I20260809 02:03:02.448313 10465 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.194:35667 every 8 connection(s)
I20260809 02:03:02.449374 10322 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786240941968412-1795-0/minicluster-data/ts-1/data/info.pb
I20260809 02:03:02.450927 10142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:02.437197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.194:56751 (local address 127.1.192.254:33057)
0809 02:03:02.437350 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0809 02:03:02.437355 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:03:02.438295 (+ 940us) server_negotiation.cc:407] Connection header received
0809 02:03:02.439243 (+ 948us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:03:02.439247 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:03:02.439308 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:03:02.439385 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:03:02.440791 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:02.441737 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:02.443224 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:02.443514 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:02.443637 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:02.446918 (+ 3281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:03:02.446921 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:03:02.446924 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:03:02.446962 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:03:02.448592 (+ 1630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:03:02.449181 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:03:02.449184 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:03:02.449185 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:03:02.449258 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:03:02.449660 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:03:02.449663 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:03:02.449664 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:03:02.449847 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:03:02.449966 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0809 02:03:02.450619 (+ 653us) server_negotiation.cc:299] Negotiation successful
0809 02:03:02.450752 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 02:03:02.451685 10349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:02.437693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:33057 (local address 127.1.192.194:56751)
0809 02:03:02.438128 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:02.438177 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 02:03:02.439028 (+ 851us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:03:02.439666 (+ 638us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:03:02.439675 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:03:02.440149 (+ 474us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:03:02.440634 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:02.440650 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:02.441881 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:02.441884 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:02.443092 (+ 1208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:02.443102 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:02.443695 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:02.443700 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:02.443896 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:02.444574 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:03:02.444602 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:03:02.446718 (+ 2116us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:03:02.448704 (+ 1986us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:03:02.448710 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:03:02.448724 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:03:02.449069 (+ 345us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:03:02.449354 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:03:02.449357 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:03:02.449359 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:03:02.449563 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:03:02.449965 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:03:02.449973 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:03:02.450348 (+ 375us) client_negotiation.cc:769] Sending connection context
0809 02:03:02.450558 (+ 210us) client_negotiation.cc:240] Negotiation successful
0809 02:03:02.450890 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":128,"threads_started":1}
I20260809 02:03:02.453001 10466 heartbeater.cc:347] Connected to a master server at 127.1.192.254:33057
I20260809 02:03:02.453205 10466 heartbeater.cc:464] Registering TS with master...
I20260809 02:03:02.453891 10466 heartbeater.cc:511] Master 127.1.192.254:33057 requested a full tablet report, sending...
I20260809 02:03:02.454974 10089 ts_manager.cc:194] Registered new tserver with Master: 8fd0ea46d91d44d685754406c9f842cf (127.1.192.194:35667)
I20260809 02:03:02.455508 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 10322
I20260809 02:03:02.455647 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786240941968412-1795-0/minicluster-data/ts-1/wal/instance
I20260809 02:03:02.455747 10089 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.192.194@KRBTEST.COM'} at 127.1.192.194:56751
I20260809 02:03:02.455963 1795 external_mini_cluster.cc:1428] Setting key d6177bf6b8a7dd26738ab839dc5ee589
I20260809 02:03:02.464116 10473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:02.455359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:33057 (local address 127.1.192.194:36789)
0809 02:03:02.456138 (+ 779us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:02.456153 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 02:03:02.456252 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:03:02.457625 (+ 1373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:03:02.457627 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:03:02.457819 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:03:02.457913 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:02.457919 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:02.458676 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:02.458680 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:02.459231 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:02.459238 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:02.459697 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:02.459700 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:02.459796 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:02.460353 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:03:02.460363 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:03:02.460703 (+ 340us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:03:02.462494 (+ 1791us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:03:02.462498 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:03:02.462501 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:03:02.462714 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:03:02.463002 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:03:02.463006 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:03:02.463008 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:03:02.463115 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:03:02.463545 (+ 430us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:03:02.463548 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:03:02.463654 (+ 106us) client_negotiation.cc:769] Sending connection context
0809 02:03:02.463763 (+ 109us) client_negotiation.cc:240] Negotiation successful
0809 02:03:02.463914 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":690,"thread_start_us":110,"threads_started":1}
I20260809 02:03:02.464108 10142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:02.455236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.194:36789 (local address 127.1.192.254:33057)
0809 02:03:02.455372 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0809 02:03:02.455376 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:03:02.456233 (+ 857us) server_negotiation.cc:407] Connection header received
0809 02:03:02.456345 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:03:02.456349 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:03:02.456398 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:03:02.456458 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:03:02.458031 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:02.458536 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:02.459362 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:02.459581 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:02.459690 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:02.460840 (+ 1150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:03:02.460845 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:03:02.460854 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:03:02.460876 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:03:02.462377 (+ 1501us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:03:02.462819 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:03:02.462824 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:03:02.462826 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:03:02.462906 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:03:02.463215 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:03:02.463219 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:03:02.463221 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:03:02.463433 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:03:02.463544 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 02:03:02.463828 (+ 284us) server_negotiation.cc:299] Negotiation successful
0809 02:03:02.463943 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
WARNING: no policy specified for oryx/127.1.192.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.192.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.192.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786240941968412-1795-0/krb5kdc/oryx_127.1.192.195.keytab.
Entry for principal oryx/127.1.192.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786240941968412-1795-0/krb5kdc/oryx_127.1.192.195.keytab.
I20260809 02:03:02.484871 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786240941968412-1795-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786240941968412-1795-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786240941968412-1795-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786240941968412-1795-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.192.195:0
--local_ip_for_outbound_sockets=127.1.192.195
--webserver_interface=127.1.192.195
--webserver_port=0
--tserver_master_addrs=127.1.192.254:33057
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786240941968412-1795-0/krb5kdc/oryx_127.1.192.195.keytab
--principal=oryx/127.1.192.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:40857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:03:02.598912 10476 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:03:02.599179 10476 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:03:02.599226 10476 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:03:02.603273 10476 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:03:02.603336 10476 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:03:02.603376 10476 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 02:03:02.603410 10476 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.192.195
I20260809 02:03:02.608610 10476 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:40857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786240941968412-1795-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786240941968412-1795-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786240941968412-1795-0/krb5kdc/oryx_127.1.192.195.keytab
--principal=oryx/127.1.192.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.192.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786240941968412-1795-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.192.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.192.254:33057
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10476
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.192.195
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786240941968412-1795-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:03:02.609717 10476 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:03:02.610720 10476 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:03:02.620992 10484 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:03:02.621033 10482 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:03:02.620992 10481 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:03:02.621300 10476 server_base.cc:1034] running on GCE node
I20260809 02:03:02.621897 10476 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:03:02.622601 10476 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:03:02.623766 10476 hybrid_clock.cc:648] HybridClock initialized: now 1786240982623745 us; error 61 us; skew 500 ppm
Aug 09 02:03:02 dist-test-slave-ss4n krb5kdc[10040](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240982, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.192.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:03:02.626690 10476 init.cc:377] Logged in from keytab as oryx/127.1.192.195@KRBTEST.COM (short username oryx)
I20260809 02:03:02.627583 10476 webserver.cc:460] Webserver started at http://127.1.192.195:41323/ using document root <none> and password file <none>
I20260809 02:03:02.628147 10476 fs_manager.cc:359] Metadata directory not provided
I20260809 02:03:02.628208 10476 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:03:02.628434 10476 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:03:02.630561 10476 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786240941968412-1795-0/minicluster-data/ts-2/data/instance:
uuid: "ae2c670022d34b0fa0ee7779e45954c9"
format_stamp: "Formatted at 2026-08-09 02:03:02 on dist-test-slave-ss4n"
server_key: "d2f18925e6af75a65726ccd618e6e522"
server_key_iv: "d905f84c1182543a33eedff1fa650689"
server_key_version: "encryptionkey@0"
I20260809 02:03:02.631142 10476 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786240941968412-1795-0/minicluster-data/ts-2/wal/instance:
uuid: "ae2c670022d34b0fa0ee7779e45954c9"
format_stamp: "Formatted at 2026-08-09 02:03:02 on dist-test-slave-ss4n"
server_key: "d2f18925e6af75a65726ccd618e6e522"
server_key_iv: "d905f84c1182543a33eedff1fa650689"
server_key_version: "encryptionkey@0"
I20260809 02:03:02.635131 10476 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 02:03:02.637756 10494 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:03:02.639008 10476 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 02:03:02.639168 10476 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786240941968412-1795-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786240941968412-1795-0/minicluster-data/ts-2/wal
uuid: "ae2c670022d34b0fa0ee7779e45954c9"
format_stamp: "Formatted at 2026-08-09 02:03:02 on dist-test-slave-ss4n"
server_key: "d2f18925e6af75a65726ccd618e6e522"
server_key_iv: "d905f84c1182543a33eedff1fa650689"
server_key_version: "encryptionkey@0"
I20260809 02:03:02.639293 10476 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786240941968412-1795-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786240941968412-1795-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786240941968412-1795-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:03:02.663856 10476 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:03:02.667748 10476 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:03:02.667964 10476 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:03:02.669456 10476 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:03:02.669582 10476 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:03:02.669667 10476 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:03:02.669710 10476 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 02:03:02 dist-test-slave-ss4n krb5kdc[10040](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240982, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.192.195@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM
I20260809 02:03:02.681336 10476 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.195:44743
I20260809 02:03:02.681418 10627 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.195:44743 every 8 connection(s)
I20260809 02:03:02.682456 10476 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786240941968412-1795-0/minicluster-data/ts-2/data/info.pb
I20260809 02:03:02.684067 10142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:02.670458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.195:55349 (local address 127.1.192.254:33057)
0809 02:03:02.670644 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0809 02:03:02.670652 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 02:03:02.671466 (+ 814us) server_negotiation.cc:407] Connection header received
0809 02:03:02.672643 (+ 1177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:03:02.672646 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:03:02.672708 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:03:02.672797 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:03:02.674336 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:02.674983 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:02.676498 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:02.676842 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:02.676967 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:02.680056 (+ 3089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:03:02.680060 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:03:02.680063 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:03:02.680094 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:03:02.681923 (+ 1829us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:03:02.682555 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:03:02.682559 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:03:02.682561 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:03:02.682652 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:03:02.682982 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:03:02.682986 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:03:02.682988 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:03:02.683231 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:03:02.683338 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 02:03:02.683835 (+ 497us) server_negotiation.cc:299] Negotiation successful
0809 02:03:02.683943 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 02:03:02.684887 10504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:02.670838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:33057 (local address 127.1.192.195:55349)
0809 02:03:02.671292 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:02.671341 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 02:03:02.672381 (+ 1040us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:03:02.673092 (+ 711us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:03:02.673102 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:03:02.673638 (+ 536us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:03:02.674186 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:02.674208 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:02.675132 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:02.675137 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:02.676364 (+ 1227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:02.676374 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:02.677026 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:02.677030 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:02.677206 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:02.677931 (+ 725us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:03:02.677948 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:03:02.679849 (+ 1901us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:03:02.682075 (+ 2226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:03:02.682087 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:03:02.682098 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:03:02.682441 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:03:02.682745 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:03:02.682747 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:03:02.682749 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:03:02.682877 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:03:02.683391 (+ 514us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:03:02.683405 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:03:02.683626 (+ 221us) client_negotiation.cc:769] Sending connection context
0809 02:03:02.683780 (+ 154us) client_negotiation.cc:240] Negotiation successful
0809 02:03:02.684024 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":134,"threads_started":1}
I20260809 02:03:02.686137 10628 heartbeater.cc:347] Connected to a master server at 127.1.192.254:33057
I20260809 02:03:02.686308 10628 heartbeater.cc:464] Registering TS with master...
I20260809 02:03:02.686867 10628 heartbeater.cc:511] Master 127.1.192.254:33057 requested a full tablet report, sending...
I20260809 02:03:02.688131 10088 ts_manager.cc:194] Registered new tserver with Master: ae2c670022d34b0fa0ee7779e45954c9 (127.1.192.195:44743)
I20260809 02:03:02.688982 10088 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.192.195@KRBTEST.COM'} at 127.1.192.195:55349
I20260809 02:03:02.692706 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 10476
I20260809 02:03:02.692852 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786240941968412-1795-0/minicluster-data/ts-2/wal/instance
I20260809 02:03:02.693192 1795 external_mini_cluster.cc:1428] Setting key f8dba30fcc855f8c7d0ce6fc32cccf08
I20260809 02:03:02.695601 1795 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 02:03:02.698957 10142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:02.688758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.195:43883 (local address 127.1.192.254:33057)
0809 02:03:02.689318 (+ 560us) server_negotiation.cc:206] Beginning negotiation
0809 02:03:02.689324 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:03:02.689339 (+ 15us) server_negotiation.cc:407] Connection header received
0809 02:03:02.689372 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:03:02.689376 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:03:02.689417 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:03:02.689540 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:03:02.690213 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:02.690759 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:02.691667 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:02.691869 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:02.691946 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:02.693196 (+ 1250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:03:02.693202 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:03:02.693208 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:03:02.693239 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:03:02.695070 (+ 1831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:03:02.697602 (+ 2532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:03:02.697608 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:03:02.697612 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:03:02.697726 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:03:02.698094 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:03:02.698099 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:03:02.698100 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:03:02.698298 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:03:02.698407 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 02:03:02.698683 (+ 276us) server_negotiation.cc:299] Negotiation successful
0809 02:03:02.698797 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":479}
I20260809 02:03:02.698971 10634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:02.688623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:33057 (local address 127.1.192.195:43883)
0809 02:03:02.688966 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:02.688980 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 02:03:02.689066 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:03:02.689673 (+ 607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:03:02.689676 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:03:02.689947 (+ 271us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:03:02.690081 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:02.690090 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:02.690894 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:02.690898 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:02.691559 (+ 661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:02.691568 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:02.691968 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:02.691971 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:02.692042 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:02.692612 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:03:02.692624 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:03:02.693061 (+ 437us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:03:02.695179 (+ 2118us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:03:02.695184 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:03:02.695186 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:03:02.695466 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:03:02.697862 (+ 2396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:03:02.697865 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:03:02.697867 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:03:02.697971 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:03:02.698409 (+ 438us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:03:02.698413 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:03:02.698515 (+ 102us) client_negotiation.cc:769] Sending connection context
0809 02:03:02.698626 (+ 111us) client_negotiation.cc:240] Negotiation successful
0809 02:03:02.698792 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":147,"threads_started":1}
Aug 09 02:03:02 dist-test-slave-ss4n krb5kdc[10040](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240982, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 02:03:02 dist-test-slave-ss4n krb5kdc[10040](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.1.192.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:03:02 dist-test-slave-ss4n krb5kdc[10040](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.1.192.254@KRBTEST.COM, Server not found in Kerberos database
W20260809 02:03:02.724246 10142 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:03:02.716493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51378 (local address 127.1.192.254:33057)
0809 02:03:02.716651 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0809 02:03:02.716655 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:03:02.716670 (+ 15us) server_negotiation.cc:407] Connection header received
0809 02:03:02.716717 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:03:02.716720 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:03:02.716763 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:03:02.716850 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:03:02.717806 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:02.718487 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:02.719595 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:02.719856 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:02.719962 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:02.724091 (+ 4129us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51378: BlockingRecv error: recv got EOF from 127.0.0.1:51378 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 09 02:03:02 dist-test-slave-ss4n krb5kdc[10040](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240982, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM
I20260809 02:03:02.734745 10142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:02.725955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51384 (local address 127.1.192.254:33057)
0809 02:03:02.726107 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0809 02:03:02.726110 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:03:02.726146 (+ 36us) server_negotiation.cc:407] Connection header received
0809 02:03:02.726289 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:03:02.726291 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:03:02.726333 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:03:02.726401 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:03:02.727111 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:02.727615 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:02.728509 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:02.728732 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:02.728810 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:02.731400 (+ 2590us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:03:02.731404 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:03:02.731409 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:03:02.731440 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:03:02.732953 (+ 1513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:03:02.733393 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:03:02.733397 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:03:02.733399 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:03:02.733532 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:03:02.733832 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:03:02.733836 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:03:02.733837 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:03:02.734024 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:03:02.734167 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0809 02:03:02.734407 (+ 240us) server_negotiation.cc:299] Negotiation successful
0809 02:03:02.734543 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 02:03:02.738054 10088 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51384:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 02:03:02.740998 10088 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:03:02.756577 10654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:02.749979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45828 (local address 127.1.192.193:35291)
0809 02:03:02.750344 (+ 365us) server_negotiation.cc:206] Beginning negotiation
0809 02:03:02.750352 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 02:03:02.750925 (+ 573us) server_negotiation.cc:407] Connection header received
0809 02:03:02.751026 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:03:02.751034 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:03:02.751284 (+ 250us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:03:02.751464 (+ 180us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:03:02.751991 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:02.753211 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:02.755221 (+ 2010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:02.755842 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:02.755992 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:02.756168 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0809 02:03:02.756298 (+ 130us) server_negotiation.cc:299] Negotiation successful
0809 02:03:02.756405 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":247,"thread_start_us":120,"threads_started":1}
I20260809 02:03:02.756587 10652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:02.749769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.193:35291 (local address 127.0.0.1:45828)
0809 02:03:02.750540 (+ 771us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:02.750597 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0809 02:03:02.750748 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:03:02.751658 (+ 910us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:03:02.751665 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:03:02.751698 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:03:02.751850 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:02.751858 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:02.753373 (+ 1515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:02.753380 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:02.755083 (+ 1703us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:02.755094 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:02.755983 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:02.755987 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:02.756102 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:02.756142 (+ 40us) client_negotiation.cc:769] Sending connection context
0809 02:03:02.756267 (+ 125us) client_negotiation.cc:240] Negotiation successful
0809 02:03:02.756406 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":567,"mutex_wait_us":294,"thread_start_us":82,"threads_started":1}
I20260809 02:03:02.756947 10653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:02.749966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.194:35667 (local address 127.0.0.1:45458)
0809 02:03:02.751198 (+ 1232us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:02.751213 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 02:03:02.751315 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:03:02.751927 (+ 612us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:03:02.751934 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:03:02.751969 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:03:02.752103 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:02.752109 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:02.753442 (+ 1333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:02.753445 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:02.755159 (+ 1714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:02.755170 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:02.756496 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:02.756499 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:02.756589 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:02.756607 (+ 18us) client_negotiation.cc:769] Sending connection context
0809 02:03:02.756702 (+ 95us) client_negotiation.cc:240] Negotiation successful
0809 02:03:02.756795 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1153,"mutex_wait_us":94,"thread_start_us":116,"threads_started":1}
I20260809 02:03:02.757265 10655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:02.750534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.195:44743 (local address 127.0.0.1:40930)
0809 02:03:02.751490 (+ 956us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:02.751504 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 02:03:02.751586 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:03:02.752103 (+ 517us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:03:02.752105 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:03:02.752122 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:03:02.752236 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:02.752243 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:02.753632 (+ 1389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:02.753635 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:02.755313 (+ 1678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:02.755321 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:02.756800 (+ 1479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:02.756804 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:02.756926 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:02.756970 (+ 44us) client_negotiation.cc:769] Sending connection context
0809 02:03:02.757058 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 02:03:02.757132 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":898,"mutex_wait_us":388,"thread_start_us":127,"threads_started":1}
I20260809 02:03:02.757723 10657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:02.750727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40930 (local address 127.1.192.195:44743)
0809 02:03:02.751202 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0809 02:03:02.751213 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 02:03:02.751689 (+ 476us) server_negotiation.cc:407] Connection header received
0809 02:03:02.751777 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:03:02.751781 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:03:02.751955 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:03:02.752110 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:03:02.752364 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:02.753534 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:02.755441 (+ 1907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:02.756187 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:02.756285 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:02.756428 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0809 02:03:02.757451 (+ 1023us) server_negotiation.cc:299] Negotiation successful
0809 02:03:02.757581 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":382,"thread_start_us":141,"threads_started":1}
I20260809 02:03:02.757570 10656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:02.750637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45458 (local address 127.1.192.194:35667)
0809 02:03:02.750936 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0809 02:03:02.750947 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 02:03:02.751327 (+ 380us) server_negotiation.cc:407] Connection header received
0809 02:03:02.751421 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:03:02.751426 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:03:02.751621 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:03:02.751776 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:03:02.752256 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:02.753214 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:02.755303 (+ 2089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:02.756129 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:02.756266 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:02.756430 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0809 02:03:02.756872 (+ 442us) server_negotiation.cc:299] Negotiation successful
0809 02:03:02.756973 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":202,"thread_start_us":114,"threads_started":1}
I20260809 02:03:02.760437 10552 tablet_service.cc:1467] Processing CreateTablet for tablet a3aa9c82dc7c450d94f91cb144194277 (DEFAULT_TABLE table=test-table [id=b1c8a60e9ceb4695bf310bbf866aedf3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:03:02.760432 10240 tablet_service.cc:1467] Processing CreateTablet for tablet a3aa9c82dc7c450d94f91cb144194277 (DEFAULT_TABLE table=test-table [id=b1c8a60e9ceb4695bf310bbf866aedf3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:03:02.761684 10552 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a3aa9c82dc7c450d94f91cb144194277. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:03:02.761831 10240 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a3aa9c82dc7c450d94f91cb144194277. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:03:02.763708 10388 tablet_service.cc:1467] Processing CreateTablet for tablet a3aa9c82dc7c450d94f91cb144194277 (DEFAULT_TABLE table=test-table [id=b1c8a60e9ceb4695bf310bbf866aedf3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:03:02.764693 10388 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a3aa9c82dc7c450d94f91cb144194277. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:03:02.771122 10658 tablet_bootstrap.cc:492] T a3aa9c82dc7c450d94f91cb144194277 P 561e9aefe338421caf224b517f925355: Bootstrap starting.
I20260809 02:03:02.771597 10660 tablet_bootstrap.cc:492] T a3aa9c82dc7c450d94f91cb144194277 P ae2c670022d34b0fa0ee7779e45954c9: Bootstrap starting.
I20260809 02:03:02.772208 10661 tablet_bootstrap.cc:492] T a3aa9c82dc7c450d94f91cb144194277 P 8fd0ea46d91d44d685754406c9f842cf: Bootstrap starting.
I20260809 02:03:02.773733 10658 tablet_bootstrap.cc:654] T a3aa9c82dc7c450d94f91cb144194277 P 561e9aefe338421caf224b517f925355: Neither blocks nor log segments found. Creating new log.
I20260809 02:03:02.774096 10660 tablet_bootstrap.cc:654] T a3aa9c82dc7c450d94f91cb144194277 P ae2c670022d34b0fa0ee7779e45954c9: Neither blocks nor log segments found. Creating new log.
I20260809 02:03:02.774421 10661 tablet_bootstrap.cc:654] T a3aa9c82dc7c450d94f91cb144194277 P 8fd0ea46d91d44d685754406c9f842cf: Neither blocks nor log segments found. Creating new log.
I20260809 02:03:02.774600 10658 log.cc:824] T a3aa9c82dc7c450d94f91cb144194277 P 561e9aefe338421caf224b517f925355: Log is configured to *not* fsync() on all Append() calls
I20260809 02:03:02.775002 10660 log.cc:824] T a3aa9c82dc7c450d94f91cb144194277 P ae2c670022d34b0fa0ee7779e45954c9: Log is configured to *not* fsync() on all Append() calls
I20260809 02:03:02.775236 10661 log.cc:824] T a3aa9c82dc7c450d94f91cb144194277 P 8fd0ea46d91d44d685754406c9f842cf: Log is configured to *not* fsync() on all Append() calls
I20260809 02:03:02.776859 10658 tablet_bootstrap.cc:492] T a3aa9c82dc7c450d94f91cb144194277 P 561e9aefe338421caf224b517f925355: No bootstrap required, opened a new log
I20260809 02:03:02.777096 10658 ts_tablet_manager.cc:1397] T a3aa9c82dc7c450d94f91cb144194277 P 561e9aefe338421caf224b517f925355: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260809 02:03:02.777570 10660 tablet_bootstrap.cc:492] T a3aa9c82dc7c450d94f91cb144194277 P ae2c670022d34b0fa0ee7779e45954c9: No bootstrap required, opened a new log
I20260809 02:03:02.777770 10660 ts_tablet_manager.cc:1397] T a3aa9c82dc7c450d94f91cb144194277 P ae2c670022d34b0fa0ee7779e45954c9: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.003s
I20260809 02:03:02.779012 10661 tablet_bootstrap.cc:492] T a3aa9c82dc7c450d94f91cb144194277 P 8fd0ea46d91d44d685754406c9f842cf: No bootstrap required, opened a new log
I20260809 02:03:02.779239 10661 ts_tablet_manager.cc:1397] T a3aa9c82dc7c450d94f91cb144194277 P 8fd0ea46d91d44d685754406c9f842cf: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260809 02:03:02.780949 10658 raft_consensus.cc:348] T a3aa9c82dc7c450d94f91cb144194277 P 561e9aefe338421caf224b517f925355 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "561e9aefe338421caf224b517f925355" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 35291 } } peers { permanent_uuid: "8fd0ea46d91d44d685754406c9f842cf" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 35667 } } peers { permanent_uuid: "ae2c670022d34b0fa0ee7779e45954c9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 44743 } }
I20260809 02:03:02.781210 10658 raft_consensus.cc:374] T a3aa9c82dc7c450d94f91cb144194277 P 561e9aefe338421caf224b517f925355 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:03:02.781279 10658 raft_consensus.cc:729] T a3aa9c82dc7c450d94f91cb144194277 P 561e9aefe338421caf224b517f925355 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 561e9aefe338421caf224b517f925355, State: Initialized, Role: FOLLOWER
I20260809 02:03:02.781807 10658 consensus_queue.cc:260] T a3aa9c82dc7c450d94f91cb144194277 P 561e9aefe338421caf224b517f925355 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "561e9aefe338421caf224b517f925355" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 35291 } } peers { permanent_uuid: "8fd0ea46d91d44d685754406c9f842cf" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 35667 } } peers { permanent_uuid: "ae2c670022d34b0fa0ee7779e45954c9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 44743 } }
I20260809 02:03:02.782073 10660 raft_consensus.cc:348] T a3aa9c82dc7c450d94f91cb144194277 P ae2c670022d34b0fa0ee7779e45954c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "561e9aefe338421caf224b517f925355" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 35291 } } peers { permanent_uuid: "8fd0ea46d91d44d685754406c9f842cf" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 35667 } } peers { permanent_uuid: "ae2c670022d34b0fa0ee7779e45954c9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 44743 } }
I20260809 02:03:02.782377 10660 raft_consensus.cc:374] T a3aa9c82dc7c450d94f91cb144194277 P ae2c670022d34b0fa0ee7779e45954c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:03:02.782454 10660 raft_consensus.cc:729] T a3aa9c82dc7c450d94f91cb144194277 P ae2c670022d34b0fa0ee7779e45954c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae2c670022d34b0fa0ee7779e45954c9, State: Initialized, Role: FOLLOWER
I20260809 02:03:02.782590 10312 heartbeater.cc:503] Master 127.1.192.254:33057 was elected leader, sending a full tablet report...
I20260809 02:03:02.782917 10658 ts_tablet_manager.cc:1428] T a3aa9c82dc7c450d94f91cb144194277 P 561e9aefe338421caf224b517f925355: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260809 02:03:02.782928 10660 consensus_queue.cc:260] T a3aa9c82dc7c450d94f91cb144194277 P ae2c670022d34b0fa0ee7779e45954c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "561e9aefe338421caf224b517f925355" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 35291 } } peers { permanent_uuid: "8fd0ea46d91d44d685754406c9f842cf" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 35667 } } peers { permanent_uuid: "ae2c670022d34b0fa0ee7779e45954c9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 44743 } }
I20260809 02:03:02.783499 10661 raft_consensus.cc:348] T a3aa9c82dc7c450d94f91cb144194277 P 8fd0ea46d91d44d685754406c9f842cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "561e9aefe338421caf224b517f925355" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 35291 } } peers { permanent_uuid: "8fd0ea46d91d44d685754406c9f842cf" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 35667 } } peers { permanent_uuid: "ae2c670022d34b0fa0ee7779e45954c9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 44743 } }
I20260809 02:03:02.783665 10628 heartbeater.cc:503] Master 127.1.192.254:33057 was elected leader, sending a full tablet report...
I20260809 02:03:02.783740 10661 raft_consensus.cc:374] T a3aa9c82dc7c450d94f91cb144194277 P 8fd0ea46d91d44d685754406c9f842cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:03:02.783792 10661 raft_consensus.cc:729] T a3aa9c82dc7c450d94f91cb144194277 P 8fd0ea46d91d44d685754406c9f842cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fd0ea46d91d44d685754406c9f842cf, State: Initialized, Role: FOLLOWER
I20260809 02:03:02.784009 10660 ts_tablet_manager.cc:1428] T a3aa9c82dc7c450d94f91cb144194277 P ae2c670022d34b0fa0ee7779e45954c9: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260809 02:03:02.784219 10661 consensus_queue.cc:260] T a3aa9c82dc7c450d94f91cb144194277 P 8fd0ea46d91d44d685754406c9f842cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "561e9aefe338421caf224b517f925355" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 35291 } } peers { permanent_uuid: "8fd0ea46d91d44d685754406c9f842cf" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 35667 } } peers { permanent_uuid: "ae2c670022d34b0fa0ee7779e45954c9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 44743 } }
I20260809 02:03:02.785159 10466 heartbeater.cc:503] Master 127.1.192.254:33057 was elected leader, sending a full tablet report...
I20260809 02:03:02.785221 10661 ts_tablet_manager.cc:1428] T a3aa9c82dc7c450d94f91cb144194277 P 8fd0ea46d91d44d685754406c9f842cf: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260809 02:03:02.793735 10665 raft_consensus.cc:482] T a3aa9c82dc7c450d94f91cb144194277 P 561e9aefe338421caf224b517f925355 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:03:02.793926 10665 raft_consensus.cc:504] T a3aa9c82dc7c450d94f91cb144194277 P 561e9aefe338421caf224b517f925355 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "561e9aefe338421caf224b517f925355" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 35291 } } peers { permanent_uuid: "8fd0ea46d91d44d685754406c9f842cf" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 35667 } } peers { permanent_uuid: "ae2c670022d34b0fa0ee7779e45954c9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 44743 } }
I20260809 02:03:02.795038 10665 leader_election.cc:288] T a3aa9c82dc7c450d94f91cb144194277 P 561e9aefe338421caf224b517f925355 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8fd0ea46d91d44d685754406c9f842cf (127.1.192.194:35667), ae2c670022d34b0fa0ee7779e45954c9 (127.1.192.195:44743)
I20260809 02:03:02.799585 10657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:02.795970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.193:45381 (local address 127.1.192.195:44743)
0809 02:03:02.796105 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0809 02:03:02.796111 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:03:02.796319 (+ 208us) server_negotiation.cc:407] Connection header received
0809 02:03:02.796439 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:03:02.796442 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:03:02.796488 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:03:02.796571 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:03:02.796849 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:02.797345 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:02.798616 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:02.799008 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:02.799138 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:02.799189 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 02:03:02.799379 (+ 190us) server_negotiation.cc:299] Negotiation successful
0809 02:03:02.799454 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 02:03:02.799613 10669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:02.795833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.195:44743 (local address 127.1.192.193:45381)
0809 02:03:02.796225 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:02.796236 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 02:03:02.796351 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:03:02.796585 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:03:02.796587 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:03:02.796603 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:03:02.796719 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:02.796733 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:02.797463 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:02.797468 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:02.798486 (+ 1018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:02.798495 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:02.799136 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:02.799139 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:02.799234 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:02.799257 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 02:03:02.799349 (+ 92us) client_negotiation.cc:240] Negotiation successful
0809 02:03:02.799466 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323,"mutex_wait_us":117,"thread_start_us":116,"threads_started":1}
I20260809 02:03:02.799746 10656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:02.795388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.193:44437 (local address 127.1.192.194:35667)
0809 02:03:02.795550 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0809 02:03:02.795556 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:03:02.795749 (+ 193us) server_negotiation.cc:407] Connection header received
0809 02:03:02.795902 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:03:02.795908 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:03:02.795986 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:03:02.796095 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:03:02.796671 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:02.797254 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:02.798647 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:02.799098 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:02.799234 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:02.799294 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 02:03:02.799514 (+ 220us) server_negotiation.cc:299] Negotiation successful
0809 02:03:02.799589 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 02:03:02.800006 10668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:02.795327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.194:35667 (local address 127.1.192.193:44437)
0809 02:03:02.795651 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:02.795665 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 02:03:02.795781 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:03:02.796387 (+ 606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:03:02.796389 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:03:02.796407 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:03:02.796531 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:02.796538 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:02.797391 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:02.797395 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:02.798509 (+ 1114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:02.798519 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:02.799227 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:02.799231 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:02.799352 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:02.799396 (+ 44us) client_negotiation.cc:769] Sending connection context
0809 02:03:02.799496 (+ 100us) client_negotiation.cc:240] Negotiation successful
0809 02:03:02.799574 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"mutex_wait_us":61,"thread_start_us":169,"threads_started":1}
I20260809 02:03:02.800259 10576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3aa9c82dc7c450d94f91cb144194277" candidate_uuid: "561e9aefe338421caf224b517f925355" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae2c670022d34b0fa0ee7779e45954c9" is_pre_election: true
I20260809 02:03:02.800658 10576 raft_consensus.cc:2454] T a3aa9c82dc7c450d94f91cb144194277 P ae2c670022d34b0fa0ee7779e45954c9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561e9aefe338421caf224b517f925355 in term 0.
I20260809 02:03:02.800806 10410 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3aa9c82dc7c450d94f91cb144194277" candidate_uuid: "561e9aefe338421caf224b517f925355" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd0ea46d91d44d685754406c9f842cf" is_pre_election: true
I20260809 02:03:02.801071 10410 raft_consensus.cc:2454] T a3aa9c82dc7c450d94f91cb144194277 P 8fd0ea46d91d44d685754406c9f842cf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561e9aefe338421caf224b517f925355 in term 0.
I20260809 02:03:02.801241 10184 leader_election.cc:302] T a3aa9c82dc7c450d94f91cb144194277 P 561e9aefe338421caf224b517f925355 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 561e9aefe338421caf224b517f925355, ae2c670022d34b0fa0ee7779e45954c9; no voters:
I20260809 02:03:02.801601 10665 raft_consensus.cc:2781] T a3aa9c82dc7c450d94f91cb144194277 P 561e9aefe338421caf224b517f925355 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:03:02.801716 10665 raft_consensus.cc:482] T a3aa9c82dc7c450d94f91cb144194277 P 561e9aefe338421caf224b517f925355 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:03:02.801815 10665 raft_consensus.cc:3037] T a3aa9c82dc7c450d94f91cb144194277 P 561e9aefe338421caf224b517f925355 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:03:02.802865 10665 raft_consensus.cc:504] T a3aa9c82dc7c450d94f91cb144194277 P 561e9aefe338421caf224b517f925355 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "561e9aefe338421caf224b517f925355" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 35291 } } peers { permanent_uuid: "8fd0ea46d91d44d685754406c9f842cf" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 35667 } } peers { permanent_uuid: "ae2c670022d34b0fa0ee7779e45954c9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 44743 } }
I20260809 02:03:02.803213 10665 leader_election.cc:288] T a3aa9c82dc7c450d94f91cb144194277 P 561e9aefe338421caf224b517f925355 [CANDIDATE]: Term 1 election: Requested vote from peers 8fd0ea46d91d44d685754406c9f842cf (127.1.192.194:35667), ae2c670022d34b0fa0ee7779e45954c9 (127.1.192.195:44743)
I20260809 02:03:02.803656 10410 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3aa9c82dc7c450d94f91cb144194277" candidate_uuid: "561e9aefe338421caf224b517f925355" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd0ea46d91d44d685754406c9f842cf"
I20260809 02:03:02.803759 10576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3aa9c82dc7c450d94f91cb144194277" candidate_uuid: "561e9aefe338421caf224b517f925355" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae2c670022d34b0fa0ee7779e45954c9"
I20260809 02:03:02.803788 10410 raft_consensus.cc:3037] T a3aa9c82dc7c450d94f91cb144194277 P 8fd0ea46d91d44d685754406c9f842cf [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:03:02.803885 10576 raft_consensus.cc:3037] T a3aa9c82dc7c450d94f91cb144194277 P ae2c670022d34b0fa0ee7779e45954c9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:03:02.805210 10576 raft_consensus.cc:2454] T a3aa9c82dc7c450d94f91cb144194277 P ae2c670022d34b0fa0ee7779e45954c9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561e9aefe338421caf224b517f925355 in term 1.
I20260809 02:03:02.805625 10184 leader_election.cc:302] T a3aa9c82dc7c450d94f91cb144194277 P 561e9aefe338421caf224b517f925355 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 561e9aefe338421caf224b517f925355, ae2c670022d34b0fa0ee7779e45954c9; no voters:
I20260809 02:03:02.805693 10410 raft_consensus.cc:2454] T a3aa9c82dc7c450d94f91cb144194277 P 8fd0ea46d91d44d685754406c9f842cf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561e9aefe338421caf224b517f925355 in term 1.
I20260809 02:03:02.805858 10665 raft_consensus.cc:2781] T a3aa9c82dc7c450d94f91cb144194277 P 561e9aefe338421caf224b517f925355 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:03:02.806109 10665 raft_consensus.cc:686] T a3aa9c82dc7c450d94f91cb144194277 P 561e9aefe338421caf224b517f925355 [term 1 LEADER]: Becoming Leader. State: Replica: 561e9aefe338421caf224b517f925355, State: Running, Role: LEADER
I20260809 02:03:02.806455 10665 consensus_queue.cc:237] T a3aa9c82dc7c450d94f91cb144194277 P 561e9aefe338421caf224b517f925355 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "561e9aefe338421caf224b517f925355" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 35291 } } peers { permanent_uuid: "8fd0ea46d91d44d685754406c9f842cf" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 35667 } } peers { permanent_uuid: "ae2c670022d34b0fa0ee7779e45954c9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 44743 } }
I20260809 02:03:02.809991 10088 catalog_manager.cc:5495] T a3aa9c82dc7c450d94f91cb144194277 P 561e9aefe338421caf224b517f925355 reported cstate change: term changed from 0 to 1, leader changed from <none> to 561e9aefe338421caf224b517f925355 (127.1.192.193). New cstate: current_term: 1 leader_uuid: "561e9aefe338421caf224b517f925355" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "561e9aefe338421caf224b517f925355" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 35291 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8fd0ea46d91d44d685754406c9f842cf" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 35667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ae2c670022d34b0fa0ee7779e45954c9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 44743 } health_report { overall_health: UNKNOWN } } }
I20260809 02:03:02.826107 10654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:02.822183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45832 (local address 127.1.192.193:35291)
0809 02:03:02.822339 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0809 02:03:02.822345 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:03:02.822359 (+ 14us) server_negotiation.cc:407] Connection header received
0809 02:03:02.822541 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:03:02.822544 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:03:02.822598 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:03:02.822742 (+ 144us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:03:02.822995 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:02.823590 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:02.824563 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:02.824802 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:02.824902 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:02.825123 (+ 221us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:03:02.825739 (+ 616us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:03:02.825842 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0809 02:03:02.825910 (+ 68us) server_negotiation.cc:299] Negotiation successful
0809 02:03:02.825990 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 02:03:02.832089 10410 raft_consensus.cc:1261] T a3aa9c82dc7c450d94f91cb144194277 P 8fd0ea46d91d44d685754406c9f842cf [term 1 FOLLOWER]: Refusing update from remote peer 561e9aefe338421caf224b517f925355: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:03:02.832080 10576 raft_consensus.cc:1261] T a3aa9c82dc7c450d94f91cb144194277 P ae2c670022d34b0fa0ee7779e45954c9 [term 1 FOLLOWER]: Refusing update from remote peer 561e9aefe338421caf224b517f925355: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:03:02.832799 10665 consensus_queue.cc:1035] T a3aa9c82dc7c450d94f91cb144194277 P 561e9aefe338421caf224b517f925355 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ae2c670022d34b0fa0ee7779e45954c9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 44743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:03:02.832988 10670 consensus_queue.cc:1035] T a3aa9c82dc7c450d94f91cb144194277 P 561e9aefe338421caf224b517f925355 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd0ea46d91d44d685754406c9f842cf" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 35667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:03:02.842938 10674 mvcc.cc:205] Tried to move back new op lower bound from 7316443065673240576 to 7316443065577803776. Current Snapshot: MvccSnapshot[applied={T|T < 7316443065673240576}]
I20260809 02:03:02.843957 10675 mvcc.cc:205] Tried to move back new op lower bound from 7316443065673240576 to 7316443065577803776. Current Snapshot: MvccSnapshot[applied={T|T < 7316443065673240576}]
I20260809 02:03:02.849653 10088 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51384:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:03:02.849844 10088 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51384:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:03:02.852347 10088 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 18e8730c5b9e4ae28cf2094606dadc7b: Sending DeleteTablet for 3 replicas of tablet a3aa9c82dc7c450d94f91cb144194277
I20260809 02:03:02.853191 10240 tablet_service.cc:1514] Processing DeleteTablet for tablet a3aa9c82dc7c450d94f91cb144194277 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:03:02 UTC) from {username='oryx', principal='oryx/127.1.192.254@KRBTEST.COM'} at 127.0.0.1:45828
I20260809 02:03:02.853402 10388 tablet_service.cc:1514] Processing DeleteTablet for tablet a3aa9c82dc7c450d94f91cb144194277 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:03:02 UTC) from {username='oryx', principal='oryx/127.1.192.254@KRBTEST.COM'} at 127.0.0.1:45458
I20260809 02:03:02.853566 10552 tablet_service.cc:1514] Processing DeleteTablet for tablet a3aa9c82dc7c450d94f91cb144194277 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:03:02 UTC) from {username='oryx', principal='oryx/127.1.192.254@KRBTEST.COM'} at 127.0.0.1:40930
I20260809 02:03:02.853946 10685 tablet_replica.cc:330] T a3aa9c82dc7c450d94f91cb144194277 P 8fd0ea46d91d44d685754406c9f842cf: stopping tablet replica
I20260809 02:03:02.854045 10686 tablet_replica.cc:330] T a3aa9c82dc7c450d94f91cb144194277 P ae2c670022d34b0fa0ee7779e45954c9: stopping tablet replica
I20260809 02:03:02.854207 10684 tablet_replica.cc:330] T a3aa9c82dc7c450d94f91cb144194277 P 561e9aefe338421caf224b517f925355: stopping tablet replica
I20260809 02:03:02.854382 10686 raft_consensus.cc:2229] T a3aa9c82dc7c450d94f91cb144194277 P ae2c670022d34b0fa0ee7779e45954c9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:03:02.854475 10684 raft_consensus.cc:2229] T a3aa9c82dc7c450d94f91cb144194277 P 561e9aefe338421caf224b517f925355 [term 1 LEADER]: Raft consensus shutting down.
I20260809 02:03:02.854710 10686 raft_consensus.cc:2258] T a3aa9c82dc7c450d94f91cb144194277 P ae2c670022d34b0fa0ee7779e45954c9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:03:02.854897 10684 raft_consensus.cc:2258] T a3aa9c82dc7c450d94f91cb144194277 P 561e9aefe338421caf224b517f925355 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:03:02.854807 10088 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51384:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 02:03:02.855281 10088 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:03:02.855736 10685 raft_consensus.cc:2229] T a3aa9c82dc7c450d94f91cb144194277 P 8fd0ea46d91d44d685754406c9f842cf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:03:02.856087 10685 raft_consensus.cc:2258] T a3aa9c82dc7c450d94f91cb144194277 P 8fd0ea46d91d44d685754406c9f842cf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:03:02.856568 10686 ts_tablet_manager.cc:1905] T a3aa9c82dc7c450d94f91cb144194277 P ae2c670022d34b0fa0ee7779e45954c9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:03:02.857285 10684 ts_tablet_manager.cc:1905] T a3aa9c82dc7c450d94f91cb144194277 P 561e9aefe338421caf224b517f925355: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:03:02.859359 10685 ts_tablet_manager.cc:1905] T a3aa9c82dc7c450d94f91cb144194277 P 8fd0ea46d91d44d685754406c9f842cf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:03:02.859508 10686 ts_tablet_manager.cc:1918] T a3aa9c82dc7c450d94f91cb144194277 P ae2c670022d34b0fa0ee7779e45954c9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:03:02.859593 10686 log.cc:1196] T a3aa9c82dc7c450d94f91cb144194277 P ae2c670022d34b0fa0ee7779e45954c9: Deleting WAL directory at /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786240941968412-1795-0/minicluster-data/ts-2/wal/wals/a3aa9c82dc7c450d94f91cb144194277
I20260809 02:03:02.859930 10686 ts_tablet_manager.cc:1939] T a3aa9c82dc7c450d94f91cb144194277 P ae2c670022d34b0fa0ee7779e45954c9: Deleting consensus metadata
I20260809 02:03:02.861228 10073 catalog_manager.cc:4841] TS ae2c670022d34b0fa0ee7779e45954c9 (127.1.192.195:44743): tablet a3aa9c82dc7c450d94f91cb144194277 (table test-table [id=b1c8a60e9ceb4695bf310bbf866aedf3]) successfully deleted
I20260809 02:03:02.861524 10388 tablet_service.cc:1467] Processing CreateTablet for tablet bf7a6131fa7a4f1eaacd776ace5f6e93 (DEFAULT_TABLE table=test-table [id=748bc0e435da4f6cb0cf4d2c2fa20859]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:03:02.861789 10388 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf7a6131fa7a4f1eaacd776ace5f6e93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:03:02.864322 10552 tablet_service.cc:1467] Processing CreateTablet for tablet bf7a6131fa7a4f1eaacd776ace5f6e93 (DEFAULT_TABLE table=test-table [id=748bc0e435da4f6cb0cf4d2c2fa20859]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:03:02.864562 10552 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf7a6131fa7a4f1eaacd776ace5f6e93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:03:02.866889 10660 tablet_bootstrap.cc:492] T bf7a6131fa7a4f1eaacd776ace5f6e93 P ae2c670022d34b0fa0ee7779e45954c9: Bootstrap starting.
I20260809 02:03:02.867884 10660 tablet_bootstrap.cc:654] T bf7a6131fa7a4f1eaacd776ace5f6e93 P ae2c670022d34b0fa0ee7779e45954c9: Neither blocks nor log segments found. Creating new log.
I20260809 02:03:02.869402 10660 tablet_bootstrap.cc:492] T bf7a6131fa7a4f1eaacd776ace5f6e93 P ae2c670022d34b0fa0ee7779e45954c9: No bootstrap required, opened a new log
I20260809 02:03:02.869525 10660 ts_tablet_manager.cc:1397] T bf7a6131fa7a4f1eaacd776ace5f6e93 P ae2c670022d34b0fa0ee7779e45954c9: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.001s
I20260809 02:03:02.870085 10660 raft_consensus.cc:348] T bf7a6131fa7a4f1eaacd776ace5f6e93 P ae2c670022d34b0fa0ee7779e45954c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "561e9aefe338421caf224b517f925355" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 35291 } } peers { permanent_uuid: "8fd0ea46d91d44d685754406c9f842cf" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 35667 } } peers { permanent_uuid: "ae2c670022d34b0fa0ee7779e45954c9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 44743 } }
I20260809 02:03:02.870211 10660 raft_consensus.cc:374] T bf7a6131fa7a4f1eaacd776ace5f6e93 P ae2c670022d34b0fa0ee7779e45954c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:03:02.870246 10660 raft_consensus.cc:729] T bf7a6131fa7a4f1eaacd776ace5f6e93 P ae2c670022d34b0fa0ee7779e45954c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae2c670022d34b0fa0ee7779e45954c9, State: Initialized, Role: FOLLOWER
I20260809 02:03:02.870316 10240 tablet_service.cc:1467] Processing CreateTablet for tablet bf7a6131fa7a4f1eaacd776ace5f6e93 (DEFAULT_TABLE table=test-table [id=748bc0e435da4f6cb0cf4d2c2fa20859]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:03:02.870374 10660 consensus_queue.cc:260] T bf7a6131fa7a4f1eaacd776ace5f6e93 P ae2c670022d34b0fa0ee7779e45954c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "561e9aefe338421caf224b517f925355" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 35291 } } peers { permanent_uuid: "8fd0ea46d91d44d685754406c9f842cf" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 35667 } } peers { permanent_uuid: "ae2c670022d34b0fa0ee7779e45954c9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 44743 } }
I20260809 02:03:02.870630 10240 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf7a6131fa7a4f1eaacd776ace5f6e93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:03:02.870651 10660 ts_tablet_manager.cc:1428] T bf7a6131fa7a4f1eaacd776ace5f6e93 P ae2c670022d34b0fa0ee7779e45954c9: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 02:03:02.871352 10684 ts_tablet_manager.cc:1918] T a3aa9c82dc7c450d94f91cb144194277 P 561e9aefe338421caf224b517f925355: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:03:02.871492 10684 log.cc:1196] T a3aa9c82dc7c450d94f91cb144194277 P 561e9aefe338421caf224b517f925355: Deleting WAL directory at /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786240941968412-1795-0/minicluster-data/ts-0/wal/wals/a3aa9c82dc7c450d94f91cb144194277
I20260809 02:03:02.871876 10684 ts_tablet_manager.cc:1939] T a3aa9c82dc7c450d94f91cb144194277 P 561e9aefe338421caf224b517f925355: Deleting consensus metadata
I20260809 02:03:02.872833 10661 tablet_bootstrap.cc:492] T bf7a6131fa7a4f1eaacd776ace5f6e93 P 8fd0ea46d91d44d685754406c9f842cf: Bootstrap starting.
I20260809 02:03:02.873201 10076 catalog_manager.cc:4841] TS 561e9aefe338421caf224b517f925355 (127.1.192.193:35291): tablet a3aa9c82dc7c450d94f91cb144194277 (table test-table [id=b1c8a60e9ceb4695bf310bbf866aedf3]) successfully deleted
I20260809 02:03:02.873682 10658 tablet_bootstrap.cc:492] T bf7a6131fa7a4f1eaacd776ace5f6e93 P 561e9aefe338421caf224b517f925355: Bootstrap starting.
I20260809 02:03:02.874037 10661 tablet_bootstrap.cc:654] T bf7a6131fa7a4f1eaacd776ace5f6e93 P 8fd0ea46d91d44d685754406c9f842cf: Neither blocks nor log segments found. Creating new log.
I20260809 02:03:02.874696 10658 tablet_bootstrap.cc:654] T bf7a6131fa7a4f1eaacd776ace5f6e93 P 561e9aefe338421caf224b517f925355: Neither blocks nor log segments found. Creating new log.
I20260809 02:03:02.875414 10685 ts_tablet_manager.cc:1918] T a3aa9c82dc7c450d94f91cb144194277 P 8fd0ea46d91d44d685754406c9f842cf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:03:02.875536 10685 log.cc:1196] T a3aa9c82dc7c450d94f91cb144194277 P 8fd0ea46d91d44d685754406c9f842cf: Deleting WAL directory at /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786240941968412-1795-0/minicluster-data/ts-1/wal/wals/a3aa9c82dc7c450d94f91cb144194277
I20260809 02:03:02.875823 10685 ts_tablet_manager.cc:1939] T a3aa9c82dc7c450d94f91cb144194277 P 8fd0ea46d91d44d685754406c9f842cf: Deleting consensus metadata
I20260809 02:03:02.875986 10661 tablet_bootstrap.cc:492] T bf7a6131fa7a4f1eaacd776ace5f6e93 P 8fd0ea46d91d44d685754406c9f842cf: No bootstrap required, opened a new log
I20260809 02:03:02.876080 10661 ts_tablet_manager.cc:1397] T bf7a6131fa7a4f1eaacd776ace5f6e93 P 8fd0ea46d91d44d685754406c9f842cf: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 02:03:02.876566 10661 raft_consensus.cc:348] T bf7a6131fa7a4f1eaacd776ace5f6e93 P 8fd0ea46d91d44d685754406c9f842cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "561e9aefe338421caf224b517f925355" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 35291 } } peers { permanent_uuid: "8fd0ea46d91d44d685754406c9f842cf" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 35667 } } peers { permanent_uuid: "ae2c670022d34b0fa0ee7779e45954c9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 44743 } }
I20260809 02:03:02.876688 10661 raft_consensus.cc:374] T bf7a6131fa7a4f1eaacd776ace5f6e93 P 8fd0ea46d91d44d685754406c9f842cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:03:02.876720 10661 raft_consensus.cc:729] T bf7a6131fa7a4f1eaacd776ace5f6e93 P 8fd0ea46d91d44d685754406c9f842cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fd0ea46d91d44d685754406c9f842cf, State: Initialized, Role: FOLLOWER
I20260809 02:03:02.876844 10661 consensus_queue.cc:260] T bf7a6131fa7a4f1eaacd776ace5f6e93 P 8fd0ea46d91d44d685754406c9f842cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "561e9aefe338421caf224b517f925355" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 35291 } } peers { permanent_uuid: "8fd0ea46d91d44d685754406c9f842cf" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 35667 } } peers { permanent_uuid: "ae2c670022d34b0fa0ee7779e45954c9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 44743 } }
I20260809 02:03:02.877064 10073 catalog_manager.cc:4841] TS 8fd0ea46d91d44d685754406c9f842cf (127.1.192.194:35667): tablet a3aa9c82dc7c450d94f91cb144194277 (table test-table [id=b1c8a60e9ceb4695bf310bbf866aedf3]) successfully deleted
I20260809 02:03:02.878194 10661 ts_tablet_manager.cc:1428] T bf7a6131fa7a4f1eaacd776ace5f6e93 P 8fd0ea46d91d44d685754406c9f842cf: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260809 02:03:02.878357 10658 tablet_bootstrap.cc:492] T bf7a6131fa7a4f1eaacd776ace5f6e93 P 561e9aefe338421caf224b517f925355: No bootstrap required, opened a new log
I20260809 02:03:02.878446 10658 ts_tablet_manager.cc:1397] T bf7a6131fa7a4f1eaacd776ace5f6e93 P 561e9aefe338421caf224b517f925355: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260809 02:03:02.878911 10658 raft_consensus.cc:348] T bf7a6131fa7a4f1eaacd776ace5f6e93 P 561e9aefe338421caf224b517f925355 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "561e9aefe338421caf224b517f925355" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 35291 } } peers { permanent_uuid: "8fd0ea46d91d44d685754406c9f842cf" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 35667 } } peers { permanent_uuid: "ae2c670022d34b0fa0ee7779e45954c9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 44743 } }
I20260809 02:03:02.879031 10658 raft_consensus.cc:374] T bf7a6131fa7a4f1eaacd776ace5f6e93 P 561e9aefe338421caf224b517f925355 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:03:02.879063 10658 raft_consensus.cc:729] T bf7a6131fa7a4f1eaacd776ace5f6e93 P 561e9aefe338421caf224b517f925355 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 561e9aefe338421caf224b517f925355, State: Initialized, Role: FOLLOWER
I20260809 02:03:02.879186 10658 consensus_queue.cc:260] T bf7a6131fa7a4f1eaacd776ace5f6e93 P 561e9aefe338421caf224b517f925355 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "561e9aefe338421caf224b517f925355" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 35291 } } peers { permanent_uuid: "8fd0ea46d91d44d685754406c9f842cf" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 35667 } } peers { permanent_uuid: "ae2c670022d34b0fa0ee7779e45954c9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 44743 } }
I20260809 02:03:02.879436 10658 ts_tablet_manager.cc:1428] T bf7a6131fa7a4f1eaacd776ace5f6e93 P 561e9aefe338421caf224b517f925355: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
W20260809 02:03:02.933889 10629 tablet.cc:2369] T bf7a6131fa7a4f1eaacd776ace5f6e93 P ae2c670022d34b0fa0ee7779e45954c9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 02:03:02.950986 10467 tablet.cc:2369] T bf7a6131fa7a4f1eaacd776ace5f6e93 P 8fd0ea46d91d44d685754406c9f842cf: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 02:03:02.969249 10313 tablet.cc:2369] T bf7a6131fa7a4f1eaacd776ace5f6e93 P 561e9aefe338421caf224b517f925355: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 02:03:02.997471 10666 raft_consensus.cc:482] T bf7a6131fa7a4f1eaacd776ace5f6e93 P ae2c670022d34b0fa0ee7779e45954c9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:03:02.997673 10666 raft_consensus.cc:504] T bf7a6131fa7a4f1eaacd776ace5f6e93 P ae2c670022d34b0fa0ee7779e45954c9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "561e9aefe338421caf224b517f925355" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 35291 } } peers { permanent_uuid: "8fd0ea46d91d44d685754406c9f842cf" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 35667 } } peers { permanent_uuid: "ae2c670022d34b0fa0ee7779e45954c9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 44743 } }
I20260809 02:03:02.998956 10666 leader_election.cc:288] T bf7a6131fa7a4f1eaacd776ace5f6e93 P ae2c670022d34b0fa0ee7779e45954c9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 561e9aefe338421caf224b517f925355 (127.1.192.193:35291), 8fd0ea46d91d44d685754406c9f842cf (127.1.192.194:35667)
I20260809 02:03:03.003628 10504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:02.999222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.194:35667 (local address 127.1.192.195:48775)
0809 02:03:02.999427 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:02.999444 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 02:03:02.999580 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:03:02.999881 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:03:02.999885 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:03:02.999908 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:03:03.000037 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:03.000043 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:03.001060 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:03.001063 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:03.002338 (+ 1275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:03.002348 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:03.003184 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:03.003188 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:03.003303 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:03.003328 (+ 25us) client_negotiation.cc:769] Sending connection context
0809 02:03:03.003412 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 02:03:03.003486 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124}
I20260809 02:03:03.003702 10656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:02.999422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.195:48775 (local address 127.1.192.194:35667)
0809 02:03:02.999557 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0809 02:03:02.999561 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:03:02.999576 (+ 15us) server_negotiation.cc:407] Connection header received
0809 02:03:02.999711 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:03:02.999714 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:03:02.999762 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:03:02.999836 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:03:03.000172 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:03.000940 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:03.002474 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:03.003051 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:03.003172 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:03.003224 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 02:03:03.003452 (+ 228us) server_negotiation.cc:299] Negotiation successful
0809 02:03:03.003558 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 02:03:03.003803 10654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:02.999376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.195:55853 (local address 127.1.192.193:35291)
0809 02:03:02.999501 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0809 02:03:02.999506 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:03:02.999935 (+ 429us) server_negotiation.cc:407] Connection header received
0809 02:03:03.000133 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:03:03.000136 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:03:03.000189 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:03:03.000265 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:03:03.000541 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:03.001214 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:03.002390 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:03.003038 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:03.003172 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:03.003224 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 02:03:03.003604 (+ 380us) server_negotiation.cc:299] Negotiation successful
0809 02:03:03.003681 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 02:03:03.004179 10691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:02.999222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.193:35291 (local address 127.1.192.195:55853)
0809 02:03:02.999847 (+ 625us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:02.999861 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 02:03:03.000009 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:03:03.000295 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:03:03.000297 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:03:03.000313 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:03:03.000422 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:03.000426 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:03.001325 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:03.001329 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:03.002252 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:03.002262 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:03.003320 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:03.003323 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:03.003454 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:03.003478 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 02:03:03.003936 (+ 458us) client_negotiation.cc:240] Negotiation successful
0809 02:03:03.004007 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":547,"mutex_wait_us":307,"thread_start_us":96,"threads_started":1}
I20260809 02:03:03.004153 10410 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf7a6131fa7a4f1eaacd776ace5f6e93" candidate_uuid: "ae2c670022d34b0fa0ee7779e45954c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd0ea46d91d44d685754406c9f842cf" is_pre_election: true
I20260809 02:03:03.004319 10410 raft_consensus.cc:2454] T bf7a6131fa7a4f1eaacd776ace5f6e93 P 8fd0ea46d91d44d685754406c9f842cf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ae2c670022d34b0fa0ee7779e45954c9 in term 0.
I20260809 02:03:03.004732 10260 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf7a6131fa7a4f1eaacd776ace5f6e93" candidate_uuid: "ae2c670022d34b0fa0ee7779e45954c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "561e9aefe338421caf224b517f925355" is_pre_election: true
I20260809 02:03:03.004724 10495 leader_election.cc:302] T bf7a6131fa7a4f1eaacd776ace5f6e93 P ae2c670022d34b0fa0ee7779e45954c9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd0ea46d91d44d685754406c9f842cf, ae2c670022d34b0fa0ee7779e45954c9; no voters:
I20260809 02:03:03.004976 10260 raft_consensus.cc:2454] T bf7a6131fa7a4f1eaacd776ace5f6e93 P 561e9aefe338421caf224b517f925355 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ae2c670022d34b0fa0ee7779e45954c9 in term 0.
I20260809 02:03:03.005035 10666 raft_consensus.cc:2781] T bf7a6131fa7a4f1eaacd776ace5f6e93 P ae2c670022d34b0fa0ee7779e45954c9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:03:03.005087 10666 raft_consensus.cc:482] T bf7a6131fa7a4f1eaacd776ace5f6e93 P ae2c670022d34b0fa0ee7779e45954c9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:03:03.005133 10666 raft_consensus.cc:3037] T bf7a6131fa7a4f1eaacd776ace5f6e93 P ae2c670022d34b0fa0ee7779e45954c9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:03:03.006265 10666 raft_consensus.cc:504] T bf7a6131fa7a4f1eaacd776ace5f6e93 P ae2c670022d34b0fa0ee7779e45954c9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "561e9aefe338421caf224b517f925355" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 35291 } } peers { permanent_uuid: "8fd0ea46d91d44d685754406c9f842cf" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 35667 } } peers { permanent_uuid: "ae2c670022d34b0fa0ee7779e45954c9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 44743 } }
I20260809 02:03:03.006661 10666 leader_election.cc:288] T bf7a6131fa7a4f1eaacd776ace5f6e93 P ae2c670022d34b0fa0ee7779e45954c9 [CANDIDATE]: Term 1 election: Requested vote from peers 561e9aefe338421caf224b517f925355 (127.1.192.193:35291), 8fd0ea46d91d44d685754406c9f842cf (127.1.192.194:35667)
I20260809 02:03:03.006996 10410 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf7a6131fa7a4f1eaacd776ace5f6e93" candidate_uuid: "ae2c670022d34b0fa0ee7779e45954c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd0ea46d91d44d685754406c9f842cf"
I20260809 02:03:03.007006 10260 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf7a6131fa7a4f1eaacd776ace5f6e93" candidate_uuid: "ae2c670022d34b0fa0ee7779e45954c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "561e9aefe338421caf224b517f925355"
I20260809 02:03:03.007118 10260 raft_consensus.cc:3037] T bf7a6131fa7a4f1eaacd776ace5f6e93 P 561e9aefe338421caf224b517f925355 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:03:03.007124 10410 raft_consensus.cc:3037] T bf7a6131fa7a4f1eaacd776ace5f6e93 P 8fd0ea46d91d44d685754406c9f842cf [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:03:03.008030 10410 raft_consensus.cc:2454] T bf7a6131fa7a4f1eaacd776ace5f6e93 P 8fd0ea46d91d44d685754406c9f842cf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ae2c670022d34b0fa0ee7779e45954c9 in term 1.
I20260809 02:03:03.008317 10495 leader_election.cc:302] T bf7a6131fa7a4f1eaacd776ace5f6e93 P ae2c670022d34b0fa0ee7779e45954c9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd0ea46d91d44d685754406c9f842cf, ae2c670022d34b0fa0ee7779e45954c9; no voters:
I20260809 02:03:03.008348 10260 raft_consensus.cc:2454] T bf7a6131fa7a4f1eaacd776ace5f6e93 P 561e9aefe338421caf224b517f925355 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ae2c670022d34b0fa0ee7779e45954c9 in term 1.
I20260809 02:03:03.008496 10666 raft_consensus.cc:2781] T bf7a6131fa7a4f1eaacd776ace5f6e93 P ae2c670022d34b0fa0ee7779e45954c9 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:03:03.008715 10666 raft_consensus.cc:686] T bf7a6131fa7a4f1eaacd776ace5f6e93 P ae2c670022d34b0fa0ee7779e45954c9 [term 1 LEADER]: Becoming Leader. State: Replica: ae2c670022d34b0fa0ee7779e45954c9, State: Running, Role: LEADER
I20260809 02:03:03.008957 10666 consensus_queue.cc:237] T bf7a6131fa7a4f1eaacd776ace5f6e93 P ae2c670022d34b0fa0ee7779e45954c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "561e9aefe338421caf224b517f925355" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 35291 } } peers { permanent_uuid: "8fd0ea46d91d44d685754406c9f842cf" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 35667 } } peers { permanent_uuid: "ae2c670022d34b0fa0ee7779e45954c9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 44743 } }
I20260809 02:03:03.011401 10087 catalog_manager.cc:5495] T bf7a6131fa7a4f1eaacd776ace5f6e93 P ae2c670022d34b0fa0ee7779e45954c9 reported cstate change: term changed from 0 to 1, leader changed from <none> to ae2c670022d34b0fa0ee7779e45954c9 (127.1.192.195). New cstate: current_term: 1 leader_uuid: "ae2c670022d34b0fa0ee7779e45954c9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "561e9aefe338421caf224b517f925355" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 35291 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8fd0ea46d91d44d685754406c9f842cf" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 35667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ae2c670022d34b0fa0ee7779e45954c9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 44743 } health_report { overall_health: HEALTHY } } }
Aug 09 02:03:03 dist-test-slave-ss4n krb5kdc[10040](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240981, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.192.254@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM
I20260809 02:03:03.057745 10700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:03.047364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:33057 (local address 127.0.0.1:51394)
0809 02:03:03.047803 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:03.047818 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 02:03:03.048003 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:03:03.048306 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:03:03.048309 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:03:03.048797 (+ 488us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:03:03.048898 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:03.048906 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:03.049899 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:03.049906 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:03.050535 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:03.050544 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:03.051028 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:03.051032 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:03.051126 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:03.051724 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:03:03.051738 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:03:03.053415 (+ 1677us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:03:03.055665 (+ 2250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:03:03.055680 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:03:03.055687 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:03:03.055971 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:03:03.056320 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:03:03.056324 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:03:03.056326 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:03:03.056432 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:03:03.056907 (+ 475us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:03:03.056917 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:03:03.057067 (+ 150us) client_negotiation.cc:769] Sending connection context
0809 02:03:03.057233 (+ 166us) client_negotiation.cc:240] Negotiation successful
0809 02:03:03.057470 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":113,"threads_started":1}
I20260809 02:03:03.057745 10142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:03.047476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51394 (local address 127.1.192.254:33057)
0809 02:03:03.047696 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0809 02:03:03.047701 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:03:03.047913 (+ 212us) server_negotiation.cc:407] Connection header received
0809 02:03:03.048119 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:03:03.048124 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:03:03.048176 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:03:03.048280 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:03:03.049050 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:03.049787 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:03.050669 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:03.050899 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:03.051020 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:03.053656 (+ 2636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:03:03.053659 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:03:03.053661 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:03:03.053683 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:03:03.055522 (+ 1839us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:03:03.056105 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:03:03.056109 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:03:03.056111 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:03:03.056207 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:03:03.056547 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:03:03.056554 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:03:03.056556 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:03:03.056771 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:03:03.056895 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0809 02:03:03.057312 (+ 417us) server_negotiation.cc:299] Negotiation successful
0809 02:03:03.057583 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":106,"server-negotiator.queue_time_us":104}
I20260809 02:03:03.062417 10087 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.1.192.254@KRBTEST.COM'} at 127.0.0.1:51394:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 02:03:03.063126 10087 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:03:03.068537 10240 tablet_service.cc:1467] Processing CreateTablet for tablet 68974bd1f4634c9b9755ab2411f32273 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ff5549a0a3a641fa9227c3812469bd5d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 02:03:03.068537 10388 tablet_service.cc:1467] Processing CreateTablet for tablet 68974bd1f4634c9b9755ab2411f32273 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ff5549a0a3a641fa9227c3812469bd5d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 02:03:03.068686 10552 tablet_service.cc:1467] Processing CreateTablet for tablet 68974bd1f4634c9b9755ab2411f32273 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ff5549a0a3a641fa9227c3812469bd5d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 02:03:03.069005 10240 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 68974bd1f4634c9b9755ab2411f32273. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:03:03.069022 10388 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 68974bd1f4634c9b9755ab2411f32273. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:03:03.069057 10552 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 68974bd1f4634c9b9755ab2411f32273. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:03:03.071483 10658 tablet_bootstrap.cc:492] T 68974bd1f4634c9b9755ab2411f32273 P 561e9aefe338421caf224b517f925355: Bootstrap starting.
I20260809 02:03:03.072217 10660 tablet_bootstrap.cc:492] T 68974bd1f4634c9b9755ab2411f32273 P ae2c670022d34b0fa0ee7779e45954c9: Bootstrap starting.
I20260809 02:03:03.072325 10658 tablet_bootstrap.cc:654] T 68974bd1f4634c9b9755ab2411f32273 P 561e9aefe338421caf224b517f925355: Neither blocks nor log segments found. Creating new log.
I20260809 02:03:03.072443 10661 tablet_bootstrap.cc:492] T 68974bd1f4634c9b9755ab2411f32273 P 8fd0ea46d91d44d685754406c9f842cf: Bootstrap starting.
I20260809 02:03:03.073212 10660 tablet_bootstrap.cc:654] T 68974bd1f4634c9b9755ab2411f32273 P ae2c670022d34b0fa0ee7779e45954c9: Neither blocks nor log segments found. Creating new log.
I20260809 02:03:03.073580 10661 tablet_bootstrap.cc:654] T 68974bd1f4634c9b9755ab2411f32273 P 8fd0ea46d91d44d685754406c9f842cf: Neither blocks nor log segments found. Creating new log.
I20260809 02:03:03.073860 10658 tablet_bootstrap.cc:492] T 68974bd1f4634c9b9755ab2411f32273 P 561e9aefe338421caf224b517f925355: No bootstrap required, opened a new log
I20260809 02:03:03.073961 10658 ts_tablet_manager.cc:1397] T 68974bd1f4634c9b9755ab2411f32273 P 561e9aefe338421caf224b517f925355: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 02:03:03.074566 10658 raft_consensus.cc:348] T 68974bd1f4634c9b9755ab2411f32273 P 561e9aefe338421caf224b517f925355 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "561e9aefe338421caf224b517f925355" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 35291 } } peers { permanent_uuid: "8fd0ea46d91d44d685754406c9f842cf" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 35667 } } peers { permanent_uuid: "ae2c670022d34b0fa0ee7779e45954c9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 44743 } }
I20260809 02:03:03.074712 10658 raft_consensus.cc:374] T 68974bd1f4634c9b9755ab2411f32273 P 561e9aefe338421caf224b517f925355 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:03:03.074754 10658 raft_consensus.cc:729] T 68974bd1f4634c9b9755ab2411f32273 P 561e9aefe338421caf224b517f925355 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 561e9aefe338421caf224b517f925355, State: Initialized, Role: FOLLOWER
I20260809 02:03:03.074884 10658 consensus_queue.cc:260] T 68974bd1f4634c9b9755ab2411f32273 P 561e9aefe338421caf224b517f925355 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "561e9aefe338421caf224b517f925355" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 35291 } } peers { permanent_uuid: "8fd0ea46d91d44d685754406c9f842cf" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 35667 } } peers { permanent_uuid: "ae2c670022d34b0fa0ee7779e45954c9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 44743 } }
I20260809 02:03:03.074976 10661 tablet_bootstrap.cc:492] T 68974bd1f4634c9b9755ab2411f32273 P 8fd0ea46d91d44d685754406c9f842cf: No bootstrap required, opened a new log
I20260809 02:03:03.075048 10661 ts_tablet_manager.cc:1397] T 68974bd1f4634c9b9755ab2411f32273 P 8fd0ea46d91d44d685754406c9f842cf: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260809 02:03:03.075193 10658 ts_tablet_manager.cc:1428] T 68974bd1f4634c9b9755ab2411f32273 P 561e9aefe338421caf224b517f925355: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 02:03:03.075147 10680 tablet_replica.cc:439] T 68974bd1f4634c9b9755ab2411f32273 P 561e9aefe338421caf224b517f925355: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "561e9aefe338421caf224b517f925355" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 35291 } } peers { permanent_uuid: "8fd0ea46d91d44d685754406c9f842cf" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 35667 } } peers { permanent_uuid: "ae2c670022d34b0fa0ee7779e45954c9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 44743 } } }
I20260809 02:03:03.075335 10680 tablet_replica.cc:442] T 68974bd1f4634c9b9755ab2411f32273 P 561e9aefe338421caf224b517f925355: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 02:03:03.075488 10661 raft_consensus.cc:348] T 68974bd1f4634c9b9755ab2411f32273 P 8fd0ea46d91d44d685754406c9f842cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "561e9aefe338421caf224b517f925355" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 35291 } } peers { permanent_uuid: "8fd0ea46d91d44d685754406c9f842cf" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 35667 } } peers { permanent_uuid: "ae2c670022d34b0fa0ee7779e45954c9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 44743 } }
I20260809 02:03:03.075579 10661 raft_consensus.cc:374] T 68974bd1f4634c9b9755ab2411f32273 P 8fd0ea46d91d44d685754406c9f842cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:03:03.075601 10661 raft_consensus.cc:729] T 68974bd1f4634c9b9755ab2411f32273 P 8fd0ea46d91d44d685754406c9f842cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fd0ea46d91d44d685754406c9f842cf, State: Initialized, Role: FOLLOWER
I20260809 02:03:03.075748 10661 consensus_queue.cc:260] T 68974bd1f4634c9b9755ab2411f32273 P 8fd0ea46d91d44d685754406c9f842cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "561e9aefe338421caf224b517f925355" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 35291 } } peers { permanent_uuid: "8fd0ea46d91d44d685754406c9f842cf" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 35667 } } peers { permanent_uuid: "ae2c670022d34b0fa0ee7779e45954c9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 44743 } }
I20260809 02:03:03.076043 10661 ts_tablet_manager.cc:1428] T 68974bd1f4634c9b9755ab2411f32273 P 8fd0ea46d91d44d685754406c9f842cf: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260809 02:03:03.076519 10660 tablet_bootstrap.cc:492] T 68974bd1f4634c9b9755ab2411f32273 P ae2c670022d34b0fa0ee7779e45954c9: No bootstrap required, opened a new log
I20260809 02:03:03.076622 10660 ts_tablet_manager.cc:1397] T 68974bd1f4634c9b9755ab2411f32273 P ae2c670022d34b0fa0ee7779e45954c9: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260809 02:03:03.077066 10660 raft_consensus.cc:348] T 68974bd1f4634c9b9755ab2411f32273 P ae2c670022d34b0fa0ee7779e45954c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "561e9aefe338421caf224b517f925355" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 35291 } } peers { permanent_uuid: "8fd0ea46d91d44d685754406c9f842cf" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 35667 } } peers { permanent_uuid: "ae2c670022d34b0fa0ee7779e45954c9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 44743 } }
I20260809 02:03:03.077236 10660 raft_consensus.cc:374] T 68974bd1f4634c9b9755ab2411f32273 P ae2c670022d34b0fa0ee7779e45954c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:03:03.077282 10660 raft_consensus.cc:729] T 68974bd1f4634c9b9755ab2411f32273 P ae2c670022d34b0fa0ee7779e45954c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae2c670022d34b0fa0ee7779e45954c9, State: Initialized, Role: FOLLOWER
I20260809 02:03:03.077425 10660 consensus_queue.cc:260] T 68974bd1f4634c9b9755ab2411f32273 P ae2c670022d34b0fa0ee7779e45954c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "561e9aefe338421caf224b517f925355" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 35291 } } peers { permanent_uuid: "8fd0ea46d91d44d685754406c9f842cf" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 35667 } } peers { permanent_uuid: "ae2c670022d34b0fa0ee7779e45954c9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 44743 } }
I20260809 02:03:03.077659 10667 tablet_replica.cc:439] T 68974bd1f4634c9b9755ab2411f32273 P 8fd0ea46d91d44d685754406c9f842cf: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "561e9aefe338421caf224b517f925355" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 35291 } } peers { permanent_uuid: "8fd0ea46d91d44d685754406c9f842cf" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 35667 } } peers { permanent_uuid: "ae2c670022d34b0fa0ee7779e45954c9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 44743 } } }
I20260809 02:03:03.077750 10660 ts_tablet_manager.cc:1428] T 68974bd1f4634c9b9755ab2411f32273 P ae2c670022d34b0fa0ee7779e45954c9: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 02:03:03.077808 10667 tablet_replica.cc:442] T 68974bd1f4634c9b9755ab2411f32273 P 8fd0ea46d91d44d685754406c9f842cf: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 02:03:03.077955 10666 tablet_replica.cc:439] T 68974bd1f4634c9b9755ab2411f32273 P ae2c670022d34b0fa0ee7779e45954c9: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "561e9aefe338421caf224b517f925355" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 35291 } } peers { permanent_uuid: "8fd0ea46d91d44d685754406c9f842cf" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 35667 } } peers { permanent_uuid: "ae2c670022d34b0fa0ee7779e45954c9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 44743 } } }
I20260809 02:03:03.078083 10666 tablet_replica.cc:442] T 68974bd1f4634c9b9755ab2411f32273 P ae2c670022d34b0fa0ee7779e45954c9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 02:03:03.250792 10667 raft_consensus.cc:482] T 68974bd1f4634c9b9755ab2411f32273 P 8fd0ea46d91d44d685754406c9f842cf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:03:03.250952 10667 raft_consensus.cc:504] T 68974bd1f4634c9b9755ab2411f32273 P 8fd0ea46d91d44d685754406c9f842cf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "561e9aefe338421caf224b517f925355" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 35291 } } peers { permanent_uuid: "8fd0ea46d91d44d685754406c9f842cf" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 35667 } } peers { permanent_uuid: "ae2c670022d34b0fa0ee7779e45954c9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 44743 } }
I20260809 02:03:03.252184 10667 leader_election.cc:288] T 68974bd1f4634c9b9755ab2411f32273 P 8fd0ea46d91d44d685754406c9f842cf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 561e9aefe338421caf224b517f925355 (127.1.192.193:35291), ae2c670022d34b0fa0ee7779e45954c9 (127.1.192.195:44743)
I20260809 02:03:03.256738 10657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:03.252698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.194:50215 (local address 127.1.192.195:44743)
0809 02:03:03.252927 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0809 02:03:03.252932 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:03:03.253091 (+ 159us) server_negotiation.cc:407] Connection header received
0809 02:03:03.253271 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:03:03.253274 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:03:03.253330 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:03:03.253442 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:03:03.253777 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:03.254396 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:03.255609 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:03.256073 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:03.256215 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:03.256270 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 02:03:03.256511 (+ 241us) server_negotiation.cc:299] Negotiation successful
0809 02:03:03.256596 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137}
I20260809 02:03:03.256760 10654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:03.252634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.194:59951 (local address 127.1.192.193:35291)
0809 02:03:03.252820 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0809 02:03:03.252827 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 02:03:03.253071 (+ 244us) server_negotiation.cc:407] Connection header received
0809 02:03:03.253303 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:03:03.253305 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:03:03.253345 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:03:03.253433 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:03:03.253830 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:03.254397 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:03.255672 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:03.256069 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:03.256214 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:03.256271 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 02:03:03.256557 (+ 286us) server_negotiation.cc:299] Negotiation successful
0809 02:03:03.256654 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 02:03:03.256769 10704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:03.252515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.195:44743 (local address 127.1.192.194:50215)
0809 02:03:03.252966 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:03.252978 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:03:03.253151 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:03:03.253454 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:03:03.253457 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:03:03.253517 (+ 60us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:03:03.253660 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:03.253668 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:03.254517 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:03.254521 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:03.255475 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:03.255484 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:03.256219 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:03.256223 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:03.256372 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:03.256397 (+ 25us) client_negotiation.cc:769] Sending connection context
0809 02:03:03.256461 (+ 64us) client_negotiation.cc:240] Negotiation successful
0809 02:03:03.256557 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":396,"mutex_wait_us":139,"spinlock_wait_cycles":37760,"thread_start_us":131,"threads_started":1}
I20260809 02:03:03.256783 10705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:03.252515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.193:35291 (local address 127.1.192.194:59951)
0809 02:03:03.252937 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:03.252955 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 02:03:03.253143 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:03:03.253454 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:03:03.253457 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:03:03.253520 (+ 63us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:03:03.253687 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:03.253693 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:03.254532 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:03.254541 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:03.255503 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:03.255513 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:03.256214 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:03.256219 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:03.256397 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:03.256415 (+ 18us) client_negotiation.cc:769] Sending connection context
0809 02:03:03.256531 (+ 116us) client_negotiation.cc:240] Negotiation successful
0809 02:03:03.256600 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":342,"mutex_wait_us":54,"spinlock_wait_cycles":45568,"thread_start_us":118,"threads_started":1}
I20260809 02:03:03.257426 10576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68974bd1f4634c9b9755ab2411f32273" candidate_uuid: "8fd0ea46d91d44d685754406c9f842cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae2c670022d34b0fa0ee7779e45954c9" is_pre_election: true
I20260809 02:03:03.257637 10576 raft_consensus.cc:2454] T 68974bd1f4634c9b9755ab2411f32273 P ae2c670022d34b0fa0ee7779e45954c9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd0ea46d91d44d685754406c9f842cf in term 0.
I20260809 02:03:03.257655 10260 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68974bd1f4634c9b9755ab2411f32273" candidate_uuid: "8fd0ea46d91d44d685754406c9f842cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "561e9aefe338421caf224b517f925355" is_pre_election: true
I20260809 02:03:03.257826 10260 raft_consensus.cc:2454] T 68974bd1f4634c9b9755ab2411f32273 P 561e9aefe338421caf224b517f925355 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd0ea46d91d44d685754406c9f842cf in term 0.
I20260809 02:03:03.258064 10341 leader_election.cc:302] T 68974bd1f4634c9b9755ab2411f32273 P 8fd0ea46d91d44d685754406c9f842cf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd0ea46d91d44d685754406c9f842cf, ae2c670022d34b0fa0ee7779e45954c9; no voters:
I20260809 02:03:03.258428 10667 raft_consensus.cc:2781] T 68974bd1f4634c9b9755ab2411f32273 P 8fd0ea46d91d44d685754406c9f842cf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:03:03.258500 10667 raft_consensus.cc:482] T 68974bd1f4634c9b9755ab2411f32273 P 8fd0ea46d91d44d685754406c9f842cf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:03:03.258544 10667 raft_consensus.cc:3037] T 68974bd1f4634c9b9755ab2411f32273 P 8fd0ea46d91d44d685754406c9f842cf [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:03:03.259656 10667 raft_consensus.cc:504] T 68974bd1f4634c9b9755ab2411f32273 P 8fd0ea46d91d44d685754406c9f842cf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "561e9aefe338421caf224b517f925355" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 35291 } } peers { permanent_uuid: "8fd0ea46d91d44d685754406c9f842cf" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 35667 } } peers { permanent_uuid: "ae2c670022d34b0fa0ee7779e45954c9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 44743 } }
I20260809 02:03:03.260021 10667 leader_election.cc:288] T 68974bd1f4634c9b9755ab2411f32273 P 8fd0ea46d91d44d685754406c9f842cf [CANDIDATE]: Term 1 election: Requested vote from peers 561e9aefe338421caf224b517f925355 (127.1.192.193:35291), ae2c670022d34b0fa0ee7779e45954c9 (127.1.192.195:44743)
I20260809 02:03:03.260324 10260 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68974bd1f4634c9b9755ab2411f32273" candidate_uuid: "8fd0ea46d91d44d685754406c9f842cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "561e9aefe338421caf224b517f925355"
I20260809 02:03:03.260456 10260 raft_consensus.cc:3037] T 68974bd1f4634c9b9755ab2411f32273 P 561e9aefe338421caf224b517f925355 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:03:03.260425 10576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68974bd1f4634c9b9755ab2411f32273" candidate_uuid: "8fd0ea46d91d44d685754406c9f842cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae2c670022d34b0fa0ee7779e45954c9"
I20260809 02:03:03.260576 10576 raft_consensus.cc:3037] T 68974bd1f4634c9b9755ab2411f32273 P ae2c670022d34b0fa0ee7779e45954c9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:03:03.261669 10260 raft_consensus.cc:2454] T 68974bd1f4634c9b9755ab2411f32273 P 561e9aefe338421caf224b517f925355 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd0ea46d91d44d685754406c9f842cf in term 1.
I20260809 02:03:03.261842 10576 raft_consensus.cc:2454] T 68974bd1f4634c9b9755ab2411f32273 P ae2c670022d34b0fa0ee7779e45954c9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd0ea46d91d44d685754406c9f842cf in term 1.
I20260809 02:03:03.261971 10344 leader_election.cc:302] T 68974bd1f4634c9b9755ab2411f32273 P 8fd0ea46d91d44d685754406c9f842cf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 561e9aefe338421caf224b517f925355, 8fd0ea46d91d44d685754406c9f842cf; no voters:
I20260809 02:03:03.262362 10667 raft_consensus.cc:2781] T 68974bd1f4634c9b9755ab2411f32273 P 8fd0ea46d91d44d685754406c9f842cf [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:03:03.262589 10667 raft_consensus.cc:686] T 68974bd1f4634c9b9755ab2411f32273 P 8fd0ea46d91d44d685754406c9f842cf [term 1 LEADER]: Becoming Leader. State: Replica: 8fd0ea46d91d44d685754406c9f842cf, State: Running, Role: LEADER
I20260809 02:03:03.262885 10667 consensus_queue.cc:237] T 68974bd1f4634c9b9755ab2411f32273 P 8fd0ea46d91d44d685754406c9f842cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "561e9aefe338421caf224b517f925355" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 35291 } } peers { permanent_uuid: "8fd0ea46d91d44d685754406c9f842cf" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 35667 } } peers { permanent_uuid: "ae2c670022d34b0fa0ee7779e45954c9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 44743 } }
I20260809 02:03:03.264317 10706 tablet_replica.cc:439] T 68974bd1f4634c9b9755ab2411f32273 P 8fd0ea46d91d44d685754406c9f842cf: TxnStatusTablet state changed. Reason: New leader 8fd0ea46d91d44d685754406c9f842cf. Latest consensus state: current_term: 1 leader_uuid: "8fd0ea46d91d44d685754406c9f842cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "561e9aefe338421caf224b517f925355" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 35291 } } peers { permanent_uuid: "8fd0ea46d91d44d685754406c9f842cf" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 35667 } } peers { permanent_uuid: "ae2c670022d34b0fa0ee7779e45954c9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 44743 } } }
I20260809 02:03:03.264446 10706 tablet_replica.cc:442] T 68974bd1f4634c9b9755ab2411f32273 P 8fd0ea46d91d44d685754406c9f842cf: This TxnStatusTablet replica's current role is: LEADER
I20260809 02:03:03.264803 10708 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 02:03:03.264973 10708 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 02:03:03.265879 10088 catalog_manager.cc:5495] T 68974bd1f4634c9b9755ab2411f32273 P 8fd0ea46d91d44d685754406c9f842cf reported cstate change: term changed from 0 to 1, leader changed from <none> to 8fd0ea46d91d44d685754406c9f842cf (127.1.192.194). New cstate: current_term: 1 leader_uuid: "8fd0ea46d91d44d685754406c9f842cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "561e9aefe338421caf224b517f925355" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 35291 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8fd0ea46d91d44d685754406c9f842cf" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 35667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ae2c670022d34b0fa0ee7779e45954c9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 44743 } health_report { overall_health: UNKNOWN } } }
I20260809 02:03:03.318784 10700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:03.314124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.194:35667 (local address 127.0.0.1:45462)
0809 02:03:03.314336 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:03.314355 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 02:03:03.314460 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:03:03.314904 (+ 444us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:03:03.314907 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:03:03.314914 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 02:03:03.315015 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:03.315019 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:03.316083 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:03.316086 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:03.316697 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:03.316705 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:03.317316 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:03.317320 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:03.317396 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:03.317446 (+ 50us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 02:03:03.318436 (+ 990us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 02:03:03.318453 (+ 17us) client_negotiation.cc:769] Sending connection context
0809 02:03:03.318561 (+ 108us) client_negotiation.cc:240] Negotiation successful
0809 02:03:03.318658 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":82}
I20260809 02:03:03.318794 10656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:03.314278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45462 (local address 127.1.192.194:35667)
0809 02:03:03.314430 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0809 02:03:03.314435 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:03:03.314453 (+ 18us) server_negotiation.cc:407] Connection header received
0809 02:03:03.314695 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:03:03.314698 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:03:03.314769 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:03:03.314870 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:03:03.315124 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:03.315961 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:03.316831 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:03.317187 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:03.317305 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:03.317727 (+ 422us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:03:03.318324 (+ 597us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:03:03.318438 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0809 02:03:03.318584 (+ 146us) server_negotiation.cc:299] Negotiation successful
0809 02:03:03.318647 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 02:03:03.322197 10576 raft_consensus.cc:1261] T 68974bd1f4634c9b9755ab2411f32273 P ae2c670022d34b0fa0ee7779e45954c9 [term 1 FOLLOWER]: Refusing update from remote peer 8fd0ea46d91d44d685754406c9f842cf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:03:03.322364 10260 raft_consensus.cc:1261] T 68974bd1f4634c9b9755ab2411f32273 P 561e9aefe338421caf224b517f925355 [term 1 FOLLOWER]: Refusing update from remote peer 8fd0ea46d91d44d685754406c9f842cf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:03:03.322922 10706 consensus_queue.cc:1035] T 68974bd1f4634c9b9755ab2411f32273 P 8fd0ea46d91d44d685754406c9f842cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ae2c670022d34b0fa0ee7779e45954c9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 44743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:03:03.323180 10667 consensus_queue.cc:1035] T 68974bd1f4634c9b9755ab2411f32273 P 8fd0ea46d91d44d685754406c9f842cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "561e9aefe338421caf224b517f925355" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 35291 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:03:03.325310 10680 tablet_replica.cc:439] T 68974bd1f4634c9b9755ab2411f32273 P 561e9aefe338421caf224b517f925355: TxnStatusTablet state changed. Reason: New leader 8fd0ea46d91d44d685754406c9f842cf. Latest consensus state: current_term: 1 leader_uuid: "8fd0ea46d91d44d685754406c9f842cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "561e9aefe338421caf224b517f925355" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 35291 } } peers { permanent_uuid: "8fd0ea46d91d44d685754406c9f842cf" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 35667 } } peers { permanent_uuid: "ae2c670022d34b0fa0ee7779e45954c9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 44743 } } }
I20260809 02:03:03.325449 10680 tablet_replica.cc:442] T 68974bd1f4634c9b9755ab2411f32273 P 561e9aefe338421caf224b517f925355: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 02:03:03.326946 10666 tablet_replica.cc:439] T 68974bd1f4634c9b9755ab2411f32273 P ae2c670022d34b0fa0ee7779e45954c9: TxnStatusTablet state changed. Reason: New leader 8fd0ea46d91d44d685754406c9f842cf. Latest consensus state: current_term: 1 leader_uuid: "8fd0ea46d91d44d685754406c9f842cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "561e9aefe338421caf224b517f925355" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 35291 } } peers { permanent_uuid: "8fd0ea46d91d44d685754406c9f842cf" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 35667 } } peers { permanent_uuid: "ae2c670022d34b0fa0ee7779e45954c9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 44743 } } }
I20260809 02:03:03.327111 10666 tablet_replica.cc:442] T 68974bd1f4634c9b9755ab2411f32273 P ae2c670022d34b0fa0ee7779e45954c9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 02:03:03.328604 10667 tablet_replica.cc:439] T 68974bd1f4634c9b9755ab2411f32273 P 8fd0ea46d91d44d685754406c9f842cf: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8fd0ea46d91d44d685754406c9f842cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "561e9aefe338421caf224b517f925355" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 35291 } } peers { permanent_uuid: "8fd0ea46d91d44d685754406c9f842cf" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 35667 } } peers { permanent_uuid: "ae2c670022d34b0fa0ee7779e45954c9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 44743 } } }
I20260809 02:03:03.328737 10667 tablet_replica.cc:442] T 68974bd1f4634c9b9755ab2411f32273 P 8fd0ea46d91d44d685754406c9f842cf: This TxnStatusTablet replica's current role is: LEADER
I20260809 02:03:03.329751 10666 tablet_replica.cc:439] T 68974bd1f4634c9b9755ab2411f32273 P ae2c670022d34b0fa0ee7779e45954c9: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8fd0ea46d91d44d685754406c9f842cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "561e9aefe338421caf224b517f925355" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 35291 } } peers { permanent_uuid: "8fd0ea46d91d44d685754406c9f842cf" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 35667 } } peers { permanent_uuid: "ae2c670022d34b0fa0ee7779e45954c9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 44743 } } }
I20260809 02:03:03.329875 10666 tablet_replica.cc:442] T 68974bd1f4634c9b9755ab2411f32273 P ae2c670022d34b0fa0ee7779e45954c9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 02:03:03.331547 10706 tablet_replica.cc:439] T 68974bd1f4634c9b9755ab2411f32273 P 8fd0ea46d91d44d685754406c9f842cf: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8fd0ea46d91d44d685754406c9f842cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "561e9aefe338421caf224b517f925355" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 35291 } } peers { permanent_uuid: "8fd0ea46d91d44d685754406c9f842cf" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 35667 } } peers { permanent_uuid: "ae2c670022d34b0fa0ee7779e45954c9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 44743 } } }
I20260809 02:03:03.331671 10706 tablet_replica.cc:442] T 68974bd1f4634c9b9755ab2411f32273 P 8fd0ea46d91d44d685754406c9f842cf: This TxnStatusTablet replica's current role is: LEADER
I20260809 02:03:03.333026 10680 tablet_replica.cc:439] T 68974bd1f4634c9b9755ab2411f32273 P 561e9aefe338421caf224b517f925355: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8fd0ea46d91d44d685754406c9f842cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "561e9aefe338421caf224b517f925355" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 35291 } } peers { permanent_uuid: "8fd0ea46d91d44d685754406c9f842cf" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 35667 } } peers { permanent_uuid: "ae2c670022d34b0fa0ee7779e45954c9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 44743 } } }
I20260809 02:03:03.333165 10680 tablet_replica.cc:442] T 68974bd1f4634c9b9755ab2411f32273 P 561e9aefe338421caf224b517f925355: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 02:03:03.333114 10673 mvcc.cc:205] Tried to move back new op lower bound from 7316443067681460224 to 7316443067447324672. Current Snapshot: MvccSnapshot[applied={T|T < 7316443067681460224}]
I20260809 02:03:03.340030 10657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:03.335708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40946 (local address 127.1.192.195:44743)
0809 02:03:03.335834 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0809 02:03:03.335838 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:03:03.335995 (+ 157us) server_negotiation.cc:407] Connection header received
0809 02:03:03.336126 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:03:03.336129 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:03:03.336197 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:03:03.336315 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:03:03.336544 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:03.337337 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:03.338348 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:03.338696 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:03.338804 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:03.339062 (+ 258us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:03:03.339645 (+ 583us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:03:03.339765 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0809 02:03:03.339823 (+ 58us) server_negotiation.cc:299] Negotiation successful
0809 02:03:03.339885 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 02:03:03.352381 10656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:03.348529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.195:55681 (local address 127.1.192.194:35667)
0809 02:03:03.348675 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0809 02:03:03.348680 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:03:03.348791 (+ 111us) server_negotiation.cc:407] Connection header received
0809 02:03:03.348915 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:03:03.348918 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:03:03.348973 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:03:03.349065 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:03:03.349307 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:03.350121 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:03.351096 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:03.351445 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:03.351554 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:03.351873 (+ 319us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:03:03.351961 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:03:03.352052 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 02:03:03.352182 (+ 130us) server_negotiation.cc:299] Negotiation successful
0809 02:03:03.352241 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 02:03:03.352370 10720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:03.348365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.194:35667 (local address 127.1.192.195:55681)
0809 02:03:03.348686 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:03.348701 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 02:03:03.348811 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:03:03.349080 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:03:03.349082 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:03:03.349090 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 02:03:03.349197 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:03.349202 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:03.350239 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:03.350242 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:03.350936 (+ 694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:03.350946 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:03.351584 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:03.351588 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:03.351675 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:03.351728 (+ 53us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 02:03:03.352073 (+ 345us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 02:03:03.352079 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 02:03:03.352147 (+ 68us) client_negotiation.cc:240] Negotiation successful
0809 02:03:03.352225 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":113,"threads_started":1}
I20260809 02:03:03.360666 10260 raft_consensus.cc:1261] T bf7a6131fa7a4f1eaacd776ace5f6e93 P 561e9aefe338421caf224b517f925355 [term 1 FOLLOWER]: Refusing update from remote peer ae2c670022d34b0fa0ee7779e45954c9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:03:03.360743 10410 raft_consensus.cc:1261] T bf7a6131fa7a4f1eaacd776ace5f6e93 P 8fd0ea46d91d44d685754406c9f842cf [term 1 FOLLOWER]: Refusing update from remote peer ae2c670022d34b0fa0ee7779e45954c9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:03:03.361177 10692 consensus_queue.cc:1035] T bf7a6131fa7a4f1eaacd776ace5f6e93 P ae2c670022d34b0fa0ee7779e45954c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561e9aefe338421caf224b517f925355" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 35291 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:03:03.361368 10666 consensus_queue.cc:1035] T bf7a6131fa7a4f1eaacd776ace5f6e93 P ae2c670022d34b0fa0ee7779e45954c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd0ea46d91d44d685754406c9f842cf" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 35667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:03:03.388244 10657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:03.384124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.194:49425 (local address 127.1.192.195:44743)
0809 02:03:03.384300 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0809 02:03:03.384307 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 02:03:03.384325 (+ 18us) server_negotiation.cc:407] Connection header received
0809 02:03:03.384384 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:03:03.384388 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:03:03.384445 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:03:03.384531 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:03:03.384843 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:03.385664 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:03.387050 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:03.387392 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:03.387494 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:03.387828 (+ 334us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:03:03.387896 (+ 68us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:03:03.387952 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 02:03:03.388083 (+ 131us) server_negotiation.cc:299] Negotiation successful
0809 02:03:03.388130 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 02:03:03.388386 10726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:03.383742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.195:44743 (local address 127.1.192.194:49425)
0809 02:03:03.384072 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:03.384091 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 02:03:03.384228 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:03:03.384561 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:03:03.384564 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:03:03.384570 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 02:03:03.384705 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:03.384711 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:03.385857 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:03.385860 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:03.386861 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:03.386875 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:03.387531 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:03.387535 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:03.387641 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:03.387696 (+ 55us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 02:03:03.387980 (+ 284us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 02:03:03.387988 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 02:03:03.388077 (+ 89us) client_negotiation.cc:240] Negotiation successful
0809 02:03:03.388164 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":93,"threads_started":1}
I20260809 02:03:03.414759 10087 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51384:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:03:03.414980 10087 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51384:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:03:03.417579 10087 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 18e8730c5b9e4ae28cf2094606dadc7b: Sending DeleteTablet for 3 replicas of tablet bf7a6131fa7a4f1eaacd776ace5f6e93
I20260809 02:03:03.418289 10240 tablet_service.cc:1514] Processing DeleteTablet for tablet bf7a6131fa7a4f1eaacd776ace5f6e93 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:03:03 UTC) from {username='oryx', principal='oryx/127.1.192.254@KRBTEST.COM'} at 127.0.0.1:45828
I20260809 02:03:03.418641 10552 tablet_service.cc:1514] Processing DeleteTablet for tablet bf7a6131fa7a4f1eaacd776ace5f6e93 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:03:03 UTC) from {username='oryx', principal='oryx/127.1.192.254@KRBTEST.COM'} at 127.0.0.1:40930
I20260809 02:03:03.419128 10730 tablet_replica.cc:330] T bf7a6131fa7a4f1eaacd776ace5f6e93 P ae2c670022d34b0fa0ee7779e45954c9: stopping tablet replica
I20260809 02:03:03.419296 10730 raft_consensus.cc:2229] T bf7a6131fa7a4f1eaacd776ace5f6e93 P ae2c670022d34b0fa0ee7779e45954c9 [term 1 LEADER]: Raft consensus shutting down.
I20260809 02:03:03.419487 10388 tablet_service.cc:1514] Processing DeleteTablet for tablet bf7a6131fa7a4f1eaacd776ace5f6e93 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:03:03 UTC) from {username='oryx', principal='oryx/127.1.192.254@KRBTEST.COM'} at 127.0.0.1:45458
I20260809 02:03:03.420362 10730 raft_consensus.cc:2258] T bf7a6131fa7a4f1eaacd776ace5f6e93 P ae2c670022d34b0fa0ee7779e45954c9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:03:03.420871 10731 tablet_replica.cc:330] T bf7a6131fa7a4f1eaacd776ace5f6e93 P 8fd0ea46d91d44d685754406c9f842cf: stopping tablet replica
I20260809 02:03:03.421077 10731 raft_consensus.cc:2229] T bf7a6131fa7a4f1eaacd776ace5f6e93 P 8fd0ea46d91d44d685754406c9f842cf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:03:03.421239 10731 raft_consensus.cc:2258] T bf7a6131fa7a4f1eaacd776ace5f6e93 P 8fd0ea46d91d44d685754406c9f842cf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:03:03.421242 10730 ts_tablet_manager.cc:1905] T bf7a6131fa7a4f1eaacd776ace5f6e93 P ae2c670022d34b0fa0ee7779e45954c9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:03:03.422248 10731 ts_tablet_manager.cc:1905] T bf7a6131fa7a4f1eaacd776ace5f6e93 P 8fd0ea46d91d44d685754406c9f842cf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:03:03.424546 10730 ts_tablet_manager.cc:1918] T bf7a6131fa7a4f1eaacd776ace5f6e93 P ae2c670022d34b0fa0ee7779e45954c9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 02:03:03.424705 10730 log.cc:1196] T bf7a6131fa7a4f1eaacd776ace5f6e93 P ae2c670022d34b0fa0ee7779e45954c9: Deleting WAL directory at /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786240941968412-1795-0/minicluster-data/ts-2/wal/wals/bf7a6131fa7a4f1eaacd776ace5f6e93
I20260809 02:03:03.425082 10730 ts_tablet_manager.cc:1939] T bf7a6131fa7a4f1eaacd776ace5f6e93 P ae2c670022d34b0fa0ee7779e45954c9: Deleting consensus metadata
I20260809 02:03:03.426030 10731 ts_tablet_manager.cc:1918] T bf7a6131fa7a4f1eaacd776ace5f6e93 P 8fd0ea46d91d44d685754406c9f842cf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 02:03:03.426010 10073 catalog_manager.cc:4841] TS ae2c670022d34b0fa0ee7779e45954c9 (127.1.192.195:44743): tablet bf7a6131fa7a4f1eaacd776ace5f6e93 (table test-table [id=748bc0e435da4f6cb0cf4d2c2fa20859]) successfully deleted
I20260809 02:03:03.426132 10731 log.cc:1196] T bf7a6131fa7a4f1eaacd776ace5f6e93 P 8fd0ea46d91d44d685754406c9f842cf: Deleting WAL directory at /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786240941968412-1795-0/minicluster-data/ts-1/wal/wals/bf7a6131fa7a4f1eaacd776ace5f6e93
I20260809 02:03:03.426482 10731 ts_tablet_manager.cc:1939] T bf7a6131fa7a4f1eaacd776ace5f6e93 P 8fd0ea46d91d44d685754406c9f842cf: Deleting consensus metadata
I20260809 02:03:03.427340 10073 catalog_manager.cc:4841] TS 8fd0ea46d91d44d685754406c9f842cf (127.1.192.194:35667): tablet bf7a6131fa7a4f1eaacd776ace5f6e93 (table test-table [id=748bc0e435da4f6cb0cf4d2c2fa20859]) successfully deleted
I20260809 02:03:03.427718 10729 tablet_replica.cc:330] T bf7a6131fa7a4f1eaacd776ace5f6e93 P 561e9aefe338421caf224b517f925355: stopping tablet replica
I20260809 02:03:03.427896 10729 raft_consensus.cc:2229] T bf7a6131fa7a4f1eaacd776ace5f6e93 P 561e9aefe338421caf224b517f925355 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:03:03.428049 10729 raft_consensus.cc:2258] T bf7a6131fa7a4f1eaacd776ace5f6e93 P 561e9aefe338421caf224b517f925355 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:03:03.428939 10729 ts_tablet_manager.cc:1905] T bf7a6131fa7a4f1eaacd776ace5f6e93 P 561e9aefe338421caf224b517f925355: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:03:03.432456 10729 ts_tablet_manager.cc:1918] T bf7a6131fa7a4f1eaacd776ace5f6e93 P 561e9aefe338421caf224b517f925355: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 02:03:03.432562 10729 log.cc:1196] T bf7a6131fa7a4f1eaacd776ace5f6e93 P 561e9aefe338421caf224b517f925355: Deleting WAL directory at /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786240941968412-1795-0/minicluster-data/ts-0/wal/wals/bf7a6131fa7a4f1eaacd776ace5f6e93
I20260809 02:03:03.432883 10729 ts_tablet_manager.cc:1939] T bf7a6131fa7a4f1eaacd776ace5f6e93 P 561e9aefe338421caf224b517f925355: Deleting consensus metadata
I20260809 02:03:03.433889 10076 catalog_manager.cc:4841] TS 561e9aefe338421caf224b517f925355 (127.1.192.193:35291): tablet bf7a6131fa7a4f1eaacd776ace5f6e93 (table test-table [id=748bc0e435da4f6cb0cf4d2c2fa20859]) successfully deleted
Aug 09 02:03:03 dist-test-slave-ss4n krb5kdc[10040](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240983, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 02:03:03 dist-test-slave-ss4n krb5kdc[10040](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.1.192.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:03:03 dist-test-slave-ss4n krb5kdc[10040](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.1.192.254@KRBTEST.COM, Server not found in Kerberos database
W20260809 02:03:03.459997 10142 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:03:03.453348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51408 (local address 127.1.192.254:33057)
0809 02:03:03.453522 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0809 02:03:03.453527 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:03:03.453770 (+ 243us) server_negotiation.cc:407] Connection header received
0809 02:03:03.453949 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:03:03.453951 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:03:03.454004 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:03:03.454100 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:03:03.454849 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:03.455394 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:03.456298 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:03.456529 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:03.456642 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:03.459888 (+ 3246us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51408: BlockingRecv error: recv got EOF from 127.0.0.1:51408 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 09 02:03:03 dist-test-slave-ss4n krb5kdc[10040](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240983, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM
I20260809 02:03:03.471606 10142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:03.462165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51424 (local address 127.1.192.254:33057)
0809 02:03:03.462302 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0809 02:03:03.462308 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:03:03.462618 (+ 310us) server_negotiation.cc:407] Connection header received
0809 02:03:03.462745 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:03:03.462748 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:03:03.462795 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:03:03.462890 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:03:03.463521 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:03.464047 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:03.464892 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:03.465131 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:03.465240 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:03.468155 (+ 2915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:03:03.468159 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:03:03.468161 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:03:03.468187 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:03:03.469758 (+ 1571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:03:03.470244 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:03:03.470247 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:03:03.470249 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:03:03.470343 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:03:03.470697 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:03:03.470702 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:03:03.470704 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:03:03.470896 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:03:03.471056 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0809 02:03:03.471316 (+ 260us) server_negotiation.cc:299] Negotiation successful
0809 02:03:03.471440 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 02:03:03.474848 10087 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51424:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 02:03:03.475348 10087 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:03:03.480672 10240 tablet_service.cc:1467] Processing CreateTablet for tablet 064732e4e2dc43b999a55b6780a025f0 (DEFAULT_TABLE table=test-table [id=2a05bb7ad05447b9b86e0089be39fb97]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:03:03.481060 10240 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 064732e4e2dc43b999a55b6780a025f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:03:03.481020 10388 tablet_service.cc:1467] Processing CreateTablet for tablet 064732e4e2dc43b999a55b6780a025f0 (DEFAULT_TABLE table=test-table [id=2a05bb7ad05447b9b86e0089be39fb97]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:03:03.481300 10388 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 064732e4e2dc43b999a55b6780a025f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:03:03.483788 10552 tablet_service.cc:1467] Processing CreateTablet for tablet 064732e4e2dc43b999a55b6780a025f0 (DEFAULT_TABLE table=test-table [id=2a05bb7ad05447b9b86e0089be39fb97]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:03:03.483886 10658 tablet_bootstrap.cc:492] T 064732e4e2dc43b999a55b6780a025f0 P 561e9aefe338421caf224b517f925355: Bootstrap starting.
I20260809 02:03:03.484089 10552 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 064732e4e2dc43b999a55b6780a025f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:03:03.484810 10658 tablet_bootstrap.cc:654] T 064732e4e2dc43b999a55b6780a025f0 P 561e9aefe338421caf224b517f925355: Neither blocks nor log segments found. Creating new log.
I20260809 02:03:03.485694 10661 tablet_bootstrap.cc:492] T 064732e4e2dc43b999a55b6780a025f0 P 8fd0ea46d91d44d685754406c9f842cf: Bootstrap starting.
I20260809 02:03:03.486610 10661 tablet_bootstrap.cc:654] T 064732e4e2dc43b999a55b6780a025f0 P 8fd0ea46d91d44d685754406c9f842cf: Neither blocks nor log segments found. Creating new log.
I20260809 02:03:03.486649 10660 tablet_bootstrap.cc:492] T 064732e4e2dc43b999a55b6780a025f0 P ae2c670022d34b0fa0ee7779e45954c9: Bootstrap starting.
I20260809 02:03:03.487820 10661 tablet_bootstrap.cc:492] T 064732e4e2dc43b999a55b6780a025f0 P 8fd0ea46d91d44d685754406c9f842cf: No bootstrap required, opened a new log
I20260809 02:03:03.487821 10660 tablet_bootstrap.cc:654] T 064732e4e2dc43b999a55b6780a025f0 P ae2c670022d34b0fa0ee7779e45954c9: Neither blocks nor log segments found. Creating new log.
I20260809 02:03:03.487934 10661 ts_tablet_manager.cc:1397] T 064732e4e2dc43b999a55b6780a025f0 P 8fd0ea46d91d44d685754406c9f842cf: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260809 02:03:03.488327 10661 raft_consensus.cc:348] T 064732e4e2dc43b999a55b6780a025f0 P 8fd0ea46d91d44d685754406c9f842cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "561e9aefe338421caf224b517f925355" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 35291 } } peers { permanent_uuid: "8fd0ea46d91d44d685754406c9f842cf" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 35667 } } peers { permanent_uuid: "ae2c670022d34b0fa0ee7779e45954c9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 44743 } }
I20260809 02:03:03.488454 10661 raft_consensus.cc:374] T 064732e4e2dc43b999a55b6780a025f0 P 8fd0ea46d91d44d685754406c9f842cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:03:03.488504 10661 raft_consensus.cc:729] T 064732e4e2dc43b999a55b6780a025f0 P 8fd0ea46d91d44d685754406c9f842cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fd0ea46d91d44d685754406c9f842cf, State: Initialized, Role: FOLLOWER
I20260809 02:03:03.488636 10661 consensus_queue.cc:260] T 064732e4e2dc43b999a55b6780a025f0 P 8fd0ea46d91d44d685754406c9f842cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "561e9aefe338421caf224b517f925355" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 35291 } } peers { permanent_uuid: "8fd0ea46d91d44d685754406c9f842cf" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 35667 } } peers { permanent_uuid: "ae2c670022d34b0fa0ee7779e45954c9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 44743 } }
I20260809 02:03:03.488919 10661 ts_tablet_manager.cc:1428] T 064732e4e2dc43b999a55b6780a025f0 P 8fd0ea46d91d44d685754406c9f842cf: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260809 02:03:03.489292 10660 tablet_bootstrap.cc:492] T 064732e4e2dc43b999a55b6780a025f0 P ae2c670022d34b0fa0ee7779e45954c9: No bootstrap required, opened a new log
I20260809 02:03:03.489392 10660 ts_tablet_manager.cc:1397] T 064732e4e2dc43b999a55b6780a025f0 P ae2c670022d34b0fa0ee7779e45954c9: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 02:03:03.489629 10658 tablet_bootstrap.cc:492] T 064732e4e2dc43b999a55b6780a025f0 P 561e9aefe338421caf224b517f925355: No bootstrap required, opened a new log
I20260809 02:03:03.489748 10658 ts_tablet_manager.cc:1397] T 064732e4e2dc43b999a55b6780a025f0 P 561e9aefe338421caf224b517f925355: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260809 02:03:03.489976 10660 raft_consensus.cc:348] T 064732e4e2dc43b999a55b6780a025f0 P ae2c670022d34b0fa0ee7779e45954c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "561e9aefe338421caf224b517f925355" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 35291 } } peers { permanent_uuid: "8fd0ea46d91d44d685754406c9f842cf" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 35667 } } peers { permanent_uuid: "ae2c670022d34b0fa0ee7779e45954c9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 44743 } }
I20260809 02:03:03.490115 10660 raft_consensus.cc:374] T 064732e4e2dc43b999a55b6780a025f0 P ae2c670022d34b0fa0ee7779e45954c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:03:03.490155 10660 raft_consensus.cc:729] T 064732e4e2dc43b999a55b6780a025f0 P ae2c670022d34b0fa0ee7779e45954c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae2c670022d34b0fa0ee7779e45954c9, State: Initialized, Role: FOLLOWER
I20260809 02:03:03.490139 10658 raft_consensus.cc:348] T 064732e4e2dc43b999a55b6780a025f0 P 561e9aefe338421caf224b517f925355 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "561e9aefe338421caf224b517f925355" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 35291 } } peers { permanent_uuid: "8fd0ea46d91d44d685754406c9f842cf" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 35667 } } peers { permanent_uuid: "ae2c670022d34b0fa0ee7779e45954c9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 44743 } }
I20260809 02:03:03.490271 10658 raft_consensus.cc:374] T 064732e4e2dc43b999a55b6780a025f0 P 561e9aefe338421caf224b517f925355 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:03:03.490310 10658 raft_consensus.cc:729] T 064732e4e2dc43b999a55b6780a025f0 P 561e9aefe338421caf224b517f925355 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 561e9aefe338421caf224b517f925355, State: Initialized, Role: FOLLOWER
I20260809 02:03:03.490304 10660 consensus_queue.cc:260] T 064732e4e2dc43b999a55b6780a025f0 P ae2c670022d34b0fa0ee7779e45954c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "561e9aefe338421caf224b517f925355" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 35291 } } peers { permanent_uuid: "8fd0ea46d91d44d685754406c9f842cf" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 35667 } } peers { permanent_uuid: "ae2c670022d34b0fa0ee7779e45954c9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 44743 } }
I20260809 02:03:03.490453 10658 consensus_queue.cc:260] T 064732e4e2dc43b999a55b6780a025f0 P 561e9aefe338421caf224b517f925355 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "561e9aefe338421caf224b517f925355" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 35291 } } peers { permanent_uuid: "8fd0ea46d91d44d685754406c9f842cf" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 35667 } } peers { permanent_uuid: "ae2c670022d34b0fa0ee7779e45954c9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 44743 } }
I20260809 02:03:03.490628 10660 ts_tablet_manager.cc:1428] T 064732e4e2dc43b999a55b6780a025f0 P ae2c670022d34b0fa0ee7779e45954c9: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 02:03:03.490741 10658 ts_tablet_manager.cc:1428] T 064732e4e2dc43b999a55b6780a025f0 P 561e9aefe338421caf224b517f925355: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 02:03:03.529027 10692 raft_consensus.cc:482] T 064732e4e2dc43b999a55b6780a025f0 P ae2c670022d34b0fa0ee7779e45954c9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:03:03.529176 10692 raft_consensus.cc:504] T 064732e4e2dc43b999a55b6780a025f0 P ae2c670022d34b0fa0ee7779e45954c9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "561e9aefe338421caf224b517f925355" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 35291 } } peers { permanent_uuid: "8fd0ea46d91d44d685754406c9f842cf" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 35667 } } peers { permanent_uuid: "ae2c670022d34b0fa0ee7779e45954c9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 44743 } }
I20260809 02:03:03.529793 10692 leader_election.cc:288] T 064732e4e2dc43b999a55b6780a025f0 P ae2c670022d34b0fa0ee7779e45954c9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 561e9aefe338421caf224b517f925355 (127.1.192.193:35291), 8fd0ea46d91d44d685754406c9f842cf (127.1.192.194:35667)
I20260809 02:03:03.530292 10260 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "064732e4e2dc43b999a55b6780a025f0" candidate_uuid: "ae2c670022d34b0fa0ee7779e45954c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "561e9aefe338421caf224b517f925355" is_pre_election: true
I20260809 02:03:03.530341 10410 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "064732e4e2dc43b999a55b6780a025f0" candidate_uuid: "ae2c670022d34b0fa0ee7779e45954c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd0ea46d91d44d685754406c9f842cf" is_pre_election: true
I20260809 02:03:03.530484 10410 raft_consensus.cc:2454] T 064732e4e2dc43b999a55b6780a025f0 P 8fd0ea46d91d44d685754406c9f842cf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ae2c670022d34b0fa0ee7779e45954c9 in term 0.
I20260809 02:03:03.530484 10260 raft_consensus.cc:2454] T 064732e4e2dc43b999a55b6780a025f0 P 561e9aefe338421caf224b517f925355 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ae2c670022d34b0fa0ee7779e45954c9 in term 0.
I20260809 02:03:03.530843 10495 leader_election.cc:302] T 064732e4e2dc43b999a55b6780a025f0 P ae2c670022d34b0fa0ee7779e45954c9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8fd0ea46d91d44d685754406c9f842cf, ae2c670022d34b0fa0ee7779e45954c9; no voters:
I20260809 02:03:03.531023 10692 raft_consensus.cc:2781] T 064732e4e2dc43b999a55b6780a025f0 P ae2c670022d34b0fa0ee7779e45954c9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:03:03.531111 10692 raft_consensus.cc:482] T 064732e4e2dc43b999a55b6780a025f0 P ae2c670022d34b0fa0ee7779e45954c9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:03:03.531172 10692 raft_consensus.cc:3037] T 064732e4e2dc43b999a55b6780a025f0 P ae2c670022d34b0fa0ee7779e45954c9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:03:03.532229 10692 raft_consensus.cc:504] T 064732e4e2dc43b999a55b6780a025f0 P ae2c670022d34b0fa0ee7779e45954c9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "561e9aefe338421caf224b517f925355" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 35291 } } peers { permanent_uuid: "8fd0ea46d91d44d685754406c9f842cf" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 35667 } } peers { permanent_uuid: "ae2c670022d34b0fa0ee7779e45954c9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 44743 } }
I20260809 02:03:03.532639 10692 leader_election.cc:288] T 064732e4e2dc43b999a55b6780a025f0 P ae2c670022d34b0fa0ee7779e45954c9 [CANDIDATE]: Term 1 election: Requested vote from peers 561e9aefe338421caf224b517f925355 (127.1.192.193:35291), 8fd0ea46d91d44d685754406c9f842cf (127.1.192.194:35667)
I20260809 02:03:03.532979 10260 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "064732e4e2dc43b999a55b6780a025f0" candidate_uuid: "ae2c670022d34b0fa0ee7779e45954c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "561e9aefe338421caf224b517f925355"
I20260809 02:03:03.533141 10260 raft_consensus.cc:3037] T 064732e4e2dc43b999a55b6780a025f0 P 561e9aefe338421caf224b517f925355 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:03:03.533080 10410 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "064732e4e2dc43b999a55b6780a025f0" candidate_uuid: "ae2c670022d34b0fa0ee7779e45954c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8fd0ea46d91d44d685754406c9f842cf"
I20260809 02:03:03.533239 10410 raft_consensus.cc:3037] T 064732e4e2dc43b999a55b6780a025f0 P 8fd0ea46d91d44d685754406c9f842cf [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:03:03.534354 10260 raft_consensus.cc:2454] T 064732e4e2dc43b999a55b6780a025f0 P 561e9aefe338421caf224b517f925355 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ae2c670022d34b0fa0ee7779e45954c9 in term 1.
I20260809 02:03:03.534578 10410 raft_consensus.cc:2454] T 064732e4e2dc43b999a55b6780a025f0 P 8fd0ea46d91d44d685754406c9f842cf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ae2c670022d34b0fa0ee7779e45954c9 in term 1.
I20260809 02:03:03.534704 10498 leader_election.cc:302] T 064732e4e2dc43b999a55b6780a025f0 P ae2c670022d34b0fa0ee7779e45954c9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 561e9aefe338421caf224b517f925355, ae2c670022d34b0fa0ee7779e45954c9; no voters:
I20260809 02:03:03.535075 10692 raft_consensus.cc:2781] T 064732e4e2dc43b999a55b6780a025f0 P ae2c670022d34b0fa0ee7779e45954c9 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:03:03.535207 10692 raft_consensus.cc:686] T 064732e4e2dc43b999a55b6780a025f0 P ae2c670022d34b0fa0ee7779e45954c9 [term 1 LEADER]: Becoming Leader. State: Replica: ae2c670022d34b0fa0ee7779e45954c9, State: Running, Role: LEADER
I20260809 02:03:03.535368 10692 consensus_queue.cc:237] T 064732e4e2dc43b999a55b6780a025f0 P ae2c670022d34b0fa0ee7779e45954c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "561e9aefe338421caf224b517f925355" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 35291 } } peers { permanent_uuid: "8fd0ea46d91d44d685754406c9f842cf" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 35667 } } peers { permanent_uuid: "ae2c670022d34b0fa0ee7779e45954c9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 44743 } }
I20260809 02:03:03.537367 10089 catalog_manager.cc:5495] T 064732e4e2dc43b999a55b6780a025f0 P ae2c670022d34b0fa0ee7779e45954c9 reported cstate change: term changed from 0 to 1, leader changed from <none> to ae2c670022d34b0fa0ee7779e45954c9 (127.1.192.195). New cstate: current_term: 1 leader_uuid: "ae2c670022d34b0fa0ee7779e45954c9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "561e9aefe338421caf224b517f925355" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 35291 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8fd0ea46d91d44d685754406c9f842cf" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 35667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ae2c670022d34b0fa0ee7779e45954c9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 44743 } health_report { overall_health: HEALTHY } } }
I20260809 02:03:03.558957 10657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:03.554692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40952 (local address 127.1.192.195:44743)
0809 02:03:03.554857 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0809 02:03:03.554861 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:03:03.554894 (+ 33us) server_negotiation.cc:407] Connection header received
0809 02:03:03.555060 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:03:03.555062 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:03:03.555110 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:03:03.555181 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:03:03.555513 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:03.556139 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:03.557432 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:03.557821 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:03.557923 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:03.558251 (+ 328us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:03:03.558331 (+ 80us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:03:03.558395 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 02:03:03.558585 (+ 190us) server_negotiation.cc:299] Negotiation successful
0809 02:03:03.558768 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 02:03:03.561239 10410 raft_consensus.cc:1261] T 064732e4e2dc43b999a55b6780a025f0 P 8fd0ea46d91d44d685754406c9f842cf [term 1 FOLLOWER]: Refusing update from remote peer ae2c670022d34b0fa0ee7779e45954c9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:03:03.561277 10260 raft_consensus.cc:1261] T 064732e4e2dc43b999a55b6780a025f0 P 561e9aefe338421caf224b517f925355 [term 1 FOLLOWER]: Refusing update from remote peer ae2c670022d34b0fa0ee7779e45954c9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:03:03.561792 10723 consensus_queue.cc:1035] T 064732e4e2dc43b999a55b6780a025f0 P ae2c670022d34b0fa0ee7779e45954c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8fd0ea46d91d44d685754406c9f842cf" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 35667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:03:03.561975 10692 consensus_queue.cc:1035] T 064732e4e2dc43b999a55b6780a025f0 P ae2c670022d34b0fa0ee7779e45954c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561e9aefe338421caf224b517f925355" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 35291 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:03:03.570765 10089 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51424:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:03:03.570933 10089 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51424:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:03:03.574303 10552 tablet_service.cc:1514] Processing DeleteTablet for tablet 064732e4e2dc43b999a55b6780a025f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:03:03 UTC) from {username='oryx', principal='oryx/127.1.192.254@KRBTEST.COM'} at 127.0.0.1:40930
I20260809 02:03:03.574546 10730 tablet_replica.cc:330] T 064732e4e2dc43b999a55b6780a025f0 P ae2c670022d34b0fa0ee7779e45954c9: stopping tablet replica
I20260809 02:03:03.574725 10730 raft_consensus.cc:2229] T 064732e4e2dc43b999a55b6780a025f0 P ae2c670022d34b0fa0ee7779e45954c9 [term 1 LEADER]: Raft consensus shutting down.
I20260809 02:03:03.574929 10730 raft_consensus.cc:2258] T 064732e4e2dc43b999a55b6780a025f0 P ae2c670022d34b0fa0ee7779e45954c9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:03:03.575048 10388 tablet_service.cc:1514] Processing DeleteTablet for tablet 064732e4e2dc43b999a55b6780a025f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:03:03 UTC) from {username='oryx', principal='oryx/127.1.192.254@KRBTEST.COM'} at 127.0.0.1:45458
I20260809 02:03:03.575179 10089 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 18e8730c5b9e4ae28cf2094606dadc7b: Sending DeleteTablet for 3 replicas of tablet 064732e4e2dc43b999a55b6780a025f0
I20260809 02:03:03.575220 10731 tablet_replica.cc:330] T 064732e4e2dc43b999a55b6780a025f0 P 8fd0ea46d91d44d685754406c9f842cf: stopping tablet replica
I20260809 02:03:03.575325 10731 raft_consensus.cc:2229] T 064732e4e2dc43b999a55b6780a025f0 P 8fd0ea46d91d44d685754406c9f842cf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:03:03.575445 10731 raft_consensus.cc:2258] T 064732e4e2dc43b999a55b6780a025f0 P 8fd0ea46d91d44d685754406c9f842cf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:03:03.575683 10240 tablet_service.cc:1514] Processing DeleteTablet for tablet 064732e4e2dc43b999a55b6780a025f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:03:03 UTC) from {username='oryx', principal='oryx/127.1.192.254@KRBTEST.COM'} at 127.0.0.1:45828
I20260809 02:03:03.575887 10729 tablet_replica.cc:330] T 064732e4e2dc43b999a55b6780a025f0 P 561e9aefe338421caf224b517f925355: stopping tablet replica
I20260809 02:03:03.576031 10729 raft_consensus.cc:2229] T 064732e4e2dc43b999a55b6780a025f0 P 561e9aefe338421caf224b517f925355 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:03:03.576053 10388 tablet_service.cc:1514] Processing DeleteTablet for tablet 064732e4e2dc43b999a55b6780a025f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:03:03 UTC) from {username='oryx', principal='oryx/127.1.192.254@KRBTEST.COM'} at 127.0.0.1:45458
I20260809 02:03:03.576066 10552 tablet_service.cc:1514] Processing DeleteTablet for tablet 064732e4e2dc43b999a55b6780a025f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:03:03 UTC) from {username='oryx', principal='oryx/127.1.192.254@KRBTEST.COM'} at 127.0.0.1:40930
I20260809 02:03:03.576179 10729 raft_consensus.cc:2258] T 064732e4e2dc43b999a55b6780a025f0 P 561e9aefe338421caf224b517f925355 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:03:03.577023 10731 ts_tablet_manager.cc:1905] T 064732e4e2dc43b999a55b6780a025f0 P 8fd0ea46d91d44d685754406c9f842cf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:03:03.577028 10730 ts_tablet_manager.cc:1905] T 064732e4e2dc43b999a55b6780a025f0 P ae2c670022d34b0fa0ee7779e45954c9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:03:03.577143 10729 ts_tablet_manager.cc:1905] T 064732e4e2dc43b999a55b6780a025f0 P 561e9aefe338421caf224b517f925355: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:03:03.579782 10731 ts_tablet_manager.cc:1918] T 064732e4e2dc43b999a55b6780a025f0 P 8fd0ea46d91d44d685754406c9f842cf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:03:03.579782 10729 ts_tablet_manager.cc:1918] T 064732e4e2dc43b999a55b6780a025f0 P 561e9aefe338421caf224b517f925355: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:03:03.579797 10730 ts_tablet_manager.cc:1918] T 064732e4e2dc43b999a55b6780a025f0 P ae2c670022d34b0fa0ee7779e45954c9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:03:03.579861 10729 log.cc:1196] T 064732e4e2dc43b999a55b6780a025f0 P 561e9aefe338421caf224b517f925355: Deleting WAL directory at /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786240941968412-1795-0/minicluster-data/ts-0/wal/wals/064732e4e2dc43b999a55b6780a025f0
I20260809 02:03:03.579861 10731 log.cc:1196] T 064732e4e2dc43b999a55b6780a025f0 P 8fd0ea46d91d44d685754406c9f842cf: Deleting WAL directory at /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786240941968412-1795-0/minicluster-data/ts-1/wal/wals/064732e4e2dc43b999a55b6780a025f0
I20260809 02:03:03.579871 10730 log.cc:1196] T 064732e4e2dc43b999a55b6780a025f0 P ae2c670022d34b0fa0ee7779e45954c9: Deleting WAL directory at /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786240941968412-1795-0/minicluster-data/ts-2/wal/wals/064732e4e2dc43b999a55b6780a025f0
I20260809 02:03:03.580166 10731 ts_tablet_manager.cc:1939] T 064732e4e2dc43b999a55b6780a025f0 P 8fd0ea46d91d44d685754406c9f842cf: Deleting consensus metadata
I20260809 02:03:03.580173 10729 ts_tablet_manager.cc:1939] T 064732e4e2dc43b999a55b6780a025f0 P 561e9aefe338421caf224b517f925355: Deleting consensus metadata
I20260809 02:03:03.580178 10730 ts_tablet_manager.cc:1939] T 064732e4e2dc43b999a55b6780a025f0 P ae2c670022d34b0fa0ee7779e45954c9: Deleting consensus metadata
I20260809 02:03:03.580904 10076 catalog_manager.cc:4841] TS 561e9aefe338421caf224b517f925355 (127.1.192.193:35291): tablet 064732e4e2dc43b999a55b6780a025f0 (table test-table [id=2a05bb7ad05447b9b86e0089be39fb97]) successfully deleted
W20260809 02:03:03.581435 10073 catalog_manager.cc:4808] TS 8fd0ea46d91d44d685754406c9f842cf (127.1.192.194:35667): delete failed for tablet 064732e4e2dc43b999a55b6780a025f0 because the tablet was not found. No further retry: Not found: Tablet not found: 064732e4e2dc43b999a55b6780a025f0
W20260809 02:03:03.582129 10073 catalog_manager.cc:4808] TS ae2c670022d34b0fa0ee7779e45954c9 (127.1.192.195:44743): delete failed for tablet 064732e4e2dc43b999a55b6780a025f0 because the tablet was not found. No further retry: Not found: Tablet not found: 064732e4e2dc43b999a55b6780a025f0
I20260809 02:03:03.582283 10089 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51424:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 02:03:03.582765 10089 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:03:03.588037 10240 tablet_service.cc:1467] Processing CreateTablet for tablet 95de13ded3eb47448650ea5325866110 (DEFAULT_TABLE table=test-table [id=8aa2864080d74089b59f0923f86154a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:03:03.588081 10388 tablet_service.cc:1467] Processing CreateTablet for tablet 95de13ded3eb47448650ea5325866110 (DEFAULT_TABLE table=test-table [id=8aa2864080d74089b59f0923f86154a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:03:03.588368 10240 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95de13ded3eb47448650ea5325866110. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:03:03.588393 10388 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95de13ded3eb47448650ea5325866110. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:03:03.590611 10658 tablet_bootstrap.cc:492] T 95de13ded3eb47448650ea5325866110 P 561e9aefe338421caf224b517f925355: Bootstrap starting.
I20260809 02:03:03.590896 10661 tablet_bootstrap.cc:492] T 95de13ded3eb47448650ea5325866110 P 8fd0ea46d91d44d685754406c9f842cf: Bootstrap starting.
I20260809 02:03:03.591236 10552 tablet_service.cc:1467] Processing CreateTablet for tablet 95de13ded3eb47448650ea5325866110 (DEFAULT_TABLE table=test-table [id=8aa2864080d74089b59f0923f86154a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:03:03.591521 10552 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95de13ded3eb47448650ea5325866110. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:03:03.591598 10658 tablet_bootstrap.cc:654] T 95de13ded3eb47448650ea5325866110 P 561e9aefe338421caf224b517f925355: Neither blocks nor log segments found. Creating new log.
I20260809 02:03:03.591739 10661 tablet_bootstrap.cc:654] T 95de13ded3eb47448650ea5325866110 P 8fd0ea46d91d44d685754406c9f842cf: Neither blocks nor log segments found. Creating new log.
I20260809 02:03:03.592938 10661 tablet_bootstrap.cc:492] T 95de13ded3eb47448650ea5325866110 P 8fd0ea46d91d44d685754406c9f842cf: No bootstrap required, opened a new log
I20260809 02:03:03.593037 10661 ts_tablet_manager.cc:1397] T 95de13ded3eb47448650ea5325866110 P 8fd0ea46d91d44d685754406c9f842cf: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 02:03:03.593467 10661 raft_consensus.cc:348] T 95de13ded3eb47448650ea5325866110 P 8fd0ea46d91d44d685754406c9f842cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "561e9aefe338421caf224b517f925355" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 35291 } } peers { permanent_uuid: "8fd0ea46d91d44d685754406c9f842cf" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 35667 } } peers { permanent_uuid: "ae2c670022d34b0fa0ee7779e45954c9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 44743 } }
I20260809 02:03:03.593636 10661 raft_consensus.cc:374] T 95de13ded3eb47448650ea5325866110 P 8fd0ea46d91d44d685754406c9f842cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:03:03.593675 10661 raft_consensus.cc:729] T 95de13ded3eb47448650ea5325866110 P 8fd0ea46d91d44d685754406c9f842cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fd0ea46d91d44d685754406c9f842cf, State: Initialized, Role: FOLLOWER
I20260809 02:03:03.593811 10661 consensus_queue.cc:260] T 95de13ded3eb47448650ea5325866110 P 8fd0ea46d91d44d685754406c9f842cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "561e9aefe338421caf224b517f925355" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 35291 } } peers { permanent_uuid: "8fd0ea46d91d44d685754406c9f842cf" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 35667 } } peers { permanent_uuid: "ae2c670022d34b0fa0ee7779e45954c9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 44743 } }
I20260809 02:03:03.594146 10661 ts_tablet_manager.cc:1428] T 95de13ded3eb47448650ea5325866110 P 8fd0ea46d91d44d685754406c9f842cf: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260809 02:03:03.594161 10660 tablet_bootstrap.cc:492] T 95de13ded3eb47448650ea5325866110 P ae2c670022d34b0fa0ee7779e45954c9: Bootstrap starting.
I20260809 02:03:03.595196 10660 tablet_bootstrap.cc:654] T 95de13ded3eb47448650ea5325866110 P ae2c670022d34b0fa0ee7779e45954c9: Neither blocks nor log segments found. Creating new log.
I20260809 02:03:03.595925 10658 tablet_bootstrap.cc:492] T 95de13ded3eb47448650ea5325866110 P 561e9aefe338421caf224b517f925355: No bootstrap required, opened a new log
I20260809 02:03:03.596057 10658 ts_tablet_manager.cc:1397] T 95de13ded3eb47448650ea5325866110 P 561e9aefe338421caf224b517f925355: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260809 02:03:03.596575 10658 raft_consensus.cc:348] T 95de13ded3eb47448650ea5325866110 P 561e9aefe338421caf224b517f925355 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "561e9aefe338421caf224b517f925355" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 35291 } } peers { permanent_uuid: "8fd0ea46d91d44d685754406c9f842cf" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 35667 } } peers { permanent_uuid: "ae2c670022d34b0fa0ee7779e45954c9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 44743 } }
I20260809 02:03:03.596736 10658 raft_consensus.cc:374] T 95de13ded3eb47448650ea5325866110 P 561e9aefe338421caf224b517f925355 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:03:03.596776 10658 raft_consensus.cc:729] T 95de13ded3eb47448650ea5325866110 P 561e9aefe338421caf224b517f925355 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 561e9aefe338421caf224b517f925355, State: Initialized, Role: FOLLOWER
I20260809 02:03:03.596807 10660 tablet_bootstrap.cc:492] T 95de13ded3eb47448650ea5325866110 P ae2c670022d34b0fa0ee7779e45954c9: No bootstrap required, opened a new log
I20260809 02:03:03.596899 10660 ts_tablet_manager.cc:1397] T 95de13ded3eb47448650ea5325866110 P ae2c670022d34b0fa0ee7779e45954c9: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 02:03:03.596915 10658 consensus_queue.cc:260] T 95de13ded3eb47448650ea5325866110 P 561e9aefe338421caf224b517f925355 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "561e9aefe338421caf224b517f925355" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 35291 } } peers { permanent_uuid: "8fd0ea46d91d44d685754406c9f842cf" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 35667 } } peers { permanent_uuid: "ae2c670022d34b0fa0ee7779e45954c9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 44743 } }
I20260809 02:03:03.597215 10658 ts_tablet_manager.cc:1428] T 95de13ded3eb47448650ea5325866110 P 561e9aefe338421caf224b517f925355: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260809 02:03:03.597404 10660 raft_consensus.cc:348] T 95de13ded3eb47448650ea5325866110 P ae2c670022d34b0fa0ee7779e45954c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "561e9aefe338421caf224b517f925355" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 35291 } } peers { permanent_uuid: "8fd0ea46d91d44d685754406c9f842cf" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 35667 } } peers { permanent_uuid: "ae2c670022d34b0fa0ee7779e45954c9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 44743 } }
I20260809 02:03:03.597606 10660 raft_consensus.cc:374] T 95de13ded3eb47448650ea5325866110 P ae2c670022d34b0fa0ee7779e45954c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:03:03.597648 10660 raft_consensus.cc:729] T 95de13ded3eb47448650ea5325866110 P ae2c670022d34b0fa0ee7779e45954c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae2c670022d34b0fa0ee7779e45954c9, State: Initialized, Role: FOLLOWER
I20260809 02:03:03.597775 10660 consensus_queue.cc:260] T 95de13ded3eb47448650ea5325866110 P ae2c670022d34b0fa0ee7779e45954c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "561e9aefe338421caf224b517f925355" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 35291 } } peers { permanent_uuid: "8fd0ea46d91d44d685754406c9f842cf" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 35667 } } peers { permanent_uuid: "ae2c670022d34b0fa0ee7779e45954c9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 44743 } }
I20260809 02:03:03.598075 10660 ts_tablet_manager.cc:1428] T 95de13ded3eb47448650ea5325866110 P ae2c670022d34b0fa0ee7779e45954c9: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 02:03:03.618178 10706 raft_consensus.cc:482] T 95de13ded3eb47448650ea5325866110 P 8fd0ea46d91d44d685754406c9f842cf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:03:03.618351 10706 raft_consensus.cc:504] T 95de13ded3eb47448650ea5325866110 P 8fd0ea46d91d44d685754406c9f842cf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "561e9aefe338421caf224b517f925355" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 35291 } } peers { permanent_uuid: "8fd0ea46d91d44d685754406c9f842cf" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 35667 } } peers { permanent_uuid: "ae2c670022d34b0fa0ee7779e45954c9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 44743 } }
I20260809 02:03:03.618886 10706 leader_election.cc:288] T 95de13ded3eb47448650ea5325866110 P 8fd0ea46d91d44d685754406c9f842cf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 561e9aefe338421caf224b517f925355 (127.1.192.193:35291), ae2c670022d34b0fa0ee7779e45954c9 (127.1.192.195:44743)
I20260809 02:03:03.619416 10576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95de13ded3eb47448650ea5325866110" candidate_uuid: "8fd0ea46d91d44d685754406c9f842cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae2c670022d34b0fa0ee7779e45954c9" is_pre_election: true
I20260809 02:03:03.619421 10260 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95de13ded3eb47448650ea5325866110" candidate_uuid: "8fd0ea46d91d44d685754406c9f842cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "561e9aefe338421caf224b517f925355" is_pre_election: true
I20260809 02:03:03.619587 10576 raft_consensus.cc:2454] T 95de13ded3eb47448650ea5325866110 P ae2c670022d34b0fa0ee7779e45954c9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd0ea46d91d44d685754406c9f842cf in term 0.
I20260809 02:03:03.619597 10260 raft_consensus.cc:2454] T 95de13ded3eb47448650ea5325866110 P 561e9aefe338421caf224b517f925355 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8fd0ea46d91d44d685754406c9f842cf in term 0.
I20260809 02:03:03.619928 10344 leader_election.cc:302] T 95de13ded3eb47448650ea5325866110 P 8fd0ea46d91d44d685754406c9f842cf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 561e9aefe338421caf224b517f925355, 8fd0ea46d91d44d685754406c9f842cf; no voters:
I20260809 02:03:03.620137 10706 raft_consensus.cc:2781] T 95de13ded3eb47448650ea5325866110 P 8fd0ea46d91d44d685754406c9f842cf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:03:03.620216 10706 raft_consensus.cc:482] T 95de13ded3eb47448650ea5325866110 P 8fd0ea46d91d44d685754406c9f842cf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:03:03.620252 10706 raft_consensus.cc:3037] T 95de13ded3eb47448650ea5325866110 P 8fd0ea46d91d44d685754406c9f842cf [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:03:03.621559 10706 raft_consensus.cc:504] T 95de13ded3eb47448650ea5325866110 P 8fd0ea46d91d44d685754406c9f842cf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "561e9aefe338421caf224b517f925355" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 35291 } } peers { permanent_uuid: "8fd0ea46d91d44d685754406c9f842cf" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 35667 } } peers { permanent_uuid: "ae2c670022d34b0fa0ee7779e45954c9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 44743 } }
I20260809 02:03:03.621913 10706 leader_election.cc:288] T 95de13ded3eb47448650ea5325866110 P 8fd0ea46d91d44d685754406c9f842cf [CANDIDATE]: Term 1 election: Requested vote from peers 561e9aefe338421caf224b517f925355 (127.1.192.193:35291), ae2c670022d34b0fa0ee7779e45954c9 (127.1.192.195:44743)
I20260809 02:03:03.622321 10260 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95de13ded3eb47448650ea5325866110" candidate_uuid: "8fd0ea46d91d44d685754406c9f842cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "561e9aefe338421caf224b517f925355"
I20260809 02:03:03.622524 10260 raft_consensus.cc:3037] T 95de13ded3eb47448650ea5325866110 P 561e9aefe338421caf224b517f925355 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:03:03.622452 10576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95de13ded3eb47448650ea5325866110" candidate_uuid: "8fd0ea46d91d44d685754406c9f842cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae2c670022d34b0fa0ee7779e45954c9"
I20260809 02:03:03.622624 10576 raft_consensus.cc:3037] T 95de13ded3eb47448650ea5325866110 P ae2c670022d34b0fa0ee7779e45954c9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:03:03.623795 10260 raft_consensus.cc:2454] T 95de13ded3eb47448650ea5325866110 P 561e9aefe338421caf224b517f925355 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd0ea46d91d44d685754406c9f842cf in term 1.
I20260809 02:03:03.623795 10576 raft_consensus.cc:2454] T 95de13ded3eb47448650ea5325866110 P ae2c670022d34b0fa0ee7779e45954c9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8fd0ea46d91d44d685754406c9f842cf in term 1.
I20260809 02:03:03.624105 10344 leader_election.cc:302] T 95de13ded3eb47448650ea5325866110 P 8fd0ea46d91d44d685754406c9f842cf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 561e9aefe338421caf224b517f925355, 8fd0ea46d91d44d685754406c9f842cf; no voters:
I20260809 02:03:03.624315 10706 raft_consensus.cc:2781] T 95de13ded3eb47448650ea5325866110 P 8fd0ea46d91d44d685754406c9f842cf [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:03:03.624388 10706 raft_consensus.cc:686] T 95de13ded3eb47448650ea5325866110 P 8fd0ea46d91d44d685754406c9f842cf [term 1 LEADER]: Becoming Leader. State: Replica: 8fd0ea46d91d44d685754406c9f842cf, State: Running, Role: LEADER
I20260809 02:03:03.624516 10706 consensus_queue.cc:237] T 95de13ded3eb47448650ea5325866110 P 8fd0ea46d91d44d685754406c9f842cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "561e9aefe338421caf224b517f925355" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 35291 } } peers { permanent_uuid: "8fd0ea46d91d44d685754406c9f842cf" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 35667 } } peers { permanent_uuid: "ae2c670022d34b0fa0ee7779e45954c9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 44743 } }
I20260809 02:03:03.626343 10088 catalog_manager.cc:5495] T 95de13ded3eb47448650ea5325866110 P 8fd0ea46d91d44d685754406c9f842cf reported cstate change: term changed from 0 to 1, leader changed from <none> to 8fd0ea46d91d44d685754406c9f842cf (127.1.192.194). New cstate: current_term: 1 leader_uuid: "8fd0ea46d91d44d685754406c9f842cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "561e9aefe338421caf224b517f925355" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 35291 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8fd0ea46d91d44d685754406c9f842cf" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 35667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ae2c670022d34b0fa0ee7779e45954c9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 44743 } health_report { overall_health: UNKNOWN } } }
I20260809 02:03:03.647840 10656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:03.644751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45470 (local address 127.1.192.194:35667)
0809 02:03:03.644859 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0809 02:03:03.644864 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:03:03.644885 (+ 21us) server_negotiation.cc:407] Connection header received
0809 02:03:03.644991 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:03:03.644993 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:03:03.645035 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:03:03.645096 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:03:03.645319 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:03.645930 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:03.646970 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:03.647181 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:03.647255 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:03.647474 (+ 219us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:03:03.647538 (+ 64us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:03:03.647590 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 02:03:03.647685 (+ 95us) server_negotiation.cc:299] Negotiation successful
0809 02:03:03.647732 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 02:03:03.657397 10656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:03.654015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.194:44385 (local address 127.1.192.194:35667)
0809 02:03:03.654156 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0809 02:03:03.654160 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:03:03.654174 (+ 14us) server_negotiation.cc:407] Connection header received
0809 02:03:03.654344 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:03:03.654346 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:03:03.654387 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:03:03.654550 (+ 163us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:03:03.654844 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:03.655397 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:03.656274 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:03.656503 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:03.656618 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:03.656921 (+ 303us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:03:03.656994 (+ 73us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:03:03.657091 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0809 02:03:03.657186 (+ 95us) server_negotiation.cc:299] Negotiation successful
0809 02:03:03.657260 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":32,"server-negotiator.queue_time_us":55}
I20260809 02:03:03.657558 10726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:03.653946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.194:35667 (local address 127.1.192.194:44385)
0809 02:03:03.654106 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:03.654123 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 02:03:03.654209 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:03:03.654567 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:03:03.654569 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:03:03.654575 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 02:03:03.654708 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:03.654713 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:03.655523 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:03.655526 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:03.656134 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:03.656143 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:03.656662 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:03.656666 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:03.656743 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:03.656773 (+ 30us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 02:03:03.657099 (+ 326us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 02:03:03.657105 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 02:03:03.657178 (+ 73us) client_negotiation.cc:240] Negotiation successful
0809 02:03:03.657300 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61,"mutex_wait_us":42}
I20260809 02:03:03.664146 10576 raft_consensus.cc:1261] T 95de13ded3eb47448650ea5325866110 P ae2c670022d34b0fa0ee7779e45954c9 [term 1 FOLLOWER]: Refusing update from remote peer 8fd0ea46d91d44d685754406c9f842cf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:03:03.664146 10260 raft_consensus.cc:1261] T 95de13ded3eb47448650ea5325866110 P 561e9aefe338421caf224b517f925355 [term 1 FOLLOWER]: Refusing update from remote peer 8fd0ea46d91d44d685754406c9f842cf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:03:03.664628 10667 consensus_queue.cc:1035] T 95de13ded3eb47448650ea5325866110 P 8fd0ea46d91d44d685754406c9f842cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "561e9aefe338421caf224b517f925355" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 35291 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:03:03.664811 10706 consensus_queue.cc:1035] T 95de13ded3eb47448650ea5325866110 P 8fd0ea46d91d44d685754406c9f842cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "ae2c670022d34b0fa0ee7779e45954c9" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 44743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:03:03.700554 10087 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51424:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:03:03.700742 10087 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51424:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:03:03.703065 10087 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 18e8730c5b9e4ae28cf2094606dadc7b: Sending DeleteTablet for 3 replicas of tablet 95de13ded3eb47448650ea5325866110
I20260809 02:03:03.703773 10240 tablet_service.cc:1514] Processing DeleteTablet for tablet 95de13ded3eb47448650ea5325866110 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:03:03 UTC) from {username='oryx', principal='oryx/127.1.192.254@KRBTEST.COM'} at 127.0.0.1:45828
I20260809 02:03:03.703999 10729 tablet_replica.cc:330] T 95de13ded3eb47448650ea5325866110 P 561e9aefe338421caf224b517f925355: stopping tablet replica
I20260809 02:03:03.704144 10729 raft_consensus.cc:2229] T 95de13ded3eb47448650ea5325866110 P 561e9aefe338421caf224b517f925355 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:03:03.704164 10390 tablet_service.cc:1514] Processing DeleteTablet for tablet 95de13ded3eb47448650ea5325866110 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:03:03 UTC) from {username='oryx', principal='oryx/127.1.192.254@KRBTEST.COM'} at 127.0.0.1:45458
I20260809 02:03:03.704265 10729 raft_consensus.cc:2258] T 95de13ded3eb47448650ea5325866110 P 561e9aefe338421caf224b517f925355 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:03:03.704361 10731 tablet_replica.cc:330] T 95de13ded3eb47448650ea5325866110 P 8fd0ea46d91d44d685754406c9f842cf: stopping tablet replica
I20260809 02:03:03.704440 10552 tablet_service.cc:1514] Processing DeleteTablet for tablet 95de13ded3eb47448650ea5325866110 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:03:03 UTC) from {username='oryx', principal='oryx/127.1.192.254@KRBTEST.COM'} at 127.0.0.1:40930
I20260809 02:03:03.704483 10731 raft_consensus.cc:2229] T 95de13ded3eb47448650ea5325866110 P 8fd0ea46d91d44d685754406c9f842cf [term 1 LEADER]: Raft consensus shutting down.
I20260809 02:03:03.704726 10730 tablet_replica.cc:330] T 95de13ded3eb47448650ea5325866110 P ae2c670022d34b0fa0ee7779e45954c9: stopping tablet replica
I20260809 02:03:03.704831 10730 raft_consensus.cc:2229] T 95de13ded3eb47448650ea5325866110 P ae2c670022d34b0fa0ee7779e45954c9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:03:03.704826 10731 raft_consensus.cc:2258] T 95de13ded3eb47448650ea5325866110 P 8fd0ea46d91d44d685754406c9f842cf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:03:03.704954 10730 raft_consensus.cc:2258] T 95de13ded3eb47448650ea5325866110 P ae2c670022d34b0fa0ee7779e45954c9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:03:03.705024 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 10166
I20260809 02:03:03.708298 10731 ts_tablet_manager.cc:1905] T 95de13ded3eb47448650ea5325866110 P 8fd0ea46d91d44d685754406c9f842cf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:03:03.710992 10731 ts_tablet_manager.cc:1918] T 95de13ded3eb47448650ea5325866110 P 8fd0ea46d91d44d685754406c9f842cf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 02:03:03.711124 10731 log.cc:1196] T 95de13ded3eb47448650ea5325866110 P 8fd0ea46d91d44d685754406c9f842cf: Deleting WAL directory at /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786240941968412-1795-0/minicluster-data/ts-1/wal/wals/95de13ded3eb47448650ea5325866110
I20260809 02:03:03.711397 10731 ts_tablet_manager.cc:1939] T 95de13ded3eb47448650ea5325866110 P 8fd0ea46d91d44d685754406c9f842cf: Deleting consensus metadata
I20260809 02:03:03.712217 10073 catalog_manager.cc:4841] TS 8fd0ea46d91d44d685754406c9f842cf (127.1.192.194:35667): tablet 95de13ded3eb47448650ea5325866110 (table test-table [id=8aa2864080d74089b59f0923f86154a1]) successfully deleted
I20260809 02:03:03.714198 10730 ts_tablet_manager.cc:1905] T 95de13ded3eb47448650ea5325866110 P ae2c670022d34b0fa0ee7779e45954c9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:03:03.716876 10730 ts_tablet_manager.cc:1918] T 95de13ded3eb47448650ea5325866110 P ae2c670022d34b0fa0ee7779e45954c9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 02:03:03.716971 10730 log.cc:1196] T 95de13ded3eb47448650ea5325866110 P ae2c670022d34b0fa0ee7779e45954c9: Deleting WAL directory at /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786240941968412-1795-0/minicluster-data/ts-2/wal/wals/95de13ded3eb47448650ea5325866110
I20260809 02:03:03.717278 10730 ts_tablet_manager.cc:1939] T 95de13ded3eb47448650ea5325866110 P ae2c670022d34b0fa0ee7779e45954c9: Deleting consensus metadata
I20260809 02:03:03.718180 10073 catalog_manager.cc:4841] TS ae2c670022d34b0fa0ee7779e45954c9 (127.1.192.195:44743): tablet 95de13ded3eb47448650ea5325866110 (table test-table [id=8aa2864080d74089b59f0923f86154a1]) successfully deleted
W20260809 02:03:03.720199 10076 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.192.193:35291 (error 108)
I20260809 02:03:03.721215 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 10322
I20260809 02:03:03.721568 10763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:03.721003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.193:35291 (local address 127.0.0.1:45842)
0809 02:03:03.721349 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:03.721431 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.193:35291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":67,"threads_started":1}
W20260809 02:03:03.721879 10076 catalog_manager.cc:4584] TS 561e9aefe338421caf224b517f925355 (127.1.192.193:35291): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 95de13ded3eb47448650ea5325866110: Network error: Client connection negotiation failed: client connection to 127.1.192.193:35291: connect: Connection refused (error 111)
I20260809 02:03:03.729790 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 10476
I20260809 02:03:03.738492 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 10056
2026-08-09T02:03:03Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3276 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786240941968412-1795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:03:03 dist-test-slave-ss4n krb5kdc[10766](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:03:03 dist-test-slave-ss4n krb5kdc[10766](info): set up 2 sockets
Aug 09 02:03:03 dist-test-slave-ss4n krb5kdc[10766](info): commencing operation
krb5kdc: starting...
W20260809 02:03:04.852051 1795 mini_kdc.cc:121] Time spent starting KDC: real 1.085s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786240941968412-1795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786240941968412-1795-0/krb5kdc/test-user.keytab.
Aug 09 02:03:04 dist-test-slave-ss4n krb5kdc[10766](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240984, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:03:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:03:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.192.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.192.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.192.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab.
Entry for principal kudu/127.1.192.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab.
I20260809 02:03:04.990883 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786240941968412-1795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786240941968412-1795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786240941968412-1795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.192.254:43293
--webserver_interface=127.1.192.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab
--principal=kudu/127.1.192.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:38875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.192.254:43293
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:03:05.106890 10781 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:03:05.107177 10781 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:03:05.107218 10781 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:03:05.111167 10781 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:03:05.111238 10781 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:03:05.111256 10781 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:03:05.111274 10781 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:03:05.111289 10781 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:03:05.116680 10781 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:38875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786240941968412-1795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786240941968412-1795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.192.254:43293
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.192.254:43293
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.192.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10781
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786240941968412-1795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:03:05.117789 10781 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:03:05.118822 10781 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:03:05.127907 10789 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:03:05.127954 10781 server_base.cc:1034] running on GCE node
W20260809 02:03:05.127975 10787 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:03:05.127907 10786 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:03:05.128773 10781 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:03:05.129969 10781 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:03:05.131187 10781 hybrid_clock.cc:648] HybridClock initialized: now 1786240985131172 us; error 33 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-09T02:03:05Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1406 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:03:05 dist-test-slave-ss4n krb5kdc[10795](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:03:05 dist-test-slave-ss4n krb5kdc[10795](info): set up 2 sockets
Aug 09 02:03:05 dist-test-slave-ss4n krb5kdc[10795](info): commencing operation
krb5kdc: starting...
W20260809 02:03:06.308845 1795 mini_kdc.cc:121] Time spent starting KDC: real 1.136s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/krb5kdc/test-user.keytab.
Aug 09 02:03:06 dist-test-slave-ss4n krb5kdc[10795](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240986, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:03:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:03:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.192.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.192.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.192.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab.
Entry for principal kudu/127.1.192.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab.
I20260809 02:03:06.446501 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.192.254:37079
--webserver_interface=127.1.192.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab
--principal=kudu/127.1.192.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:41989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.192.254:37079
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:03:06.562959 10809 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:03:06.563331 10809 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:03:06.563421 10809 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:03:06.567801 10809 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:03:06.567910 10809 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:03:06.567950 10809 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:03:06.567987 10809 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:03:06.568020 10809 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:03:06.573676 10809 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:41989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.192.254:37079
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab
--principal=kudu/127.1.192.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.192.254:37079
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.192.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10809
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:03:06.574798 10809 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:03:06.575840 10809 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:03:06.584784 10815 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:03:06.584794 10818 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:03:06.584936 10809 server_base.cc:1034] running on GCE node
W20260809 02:03:06.584836 10816 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:03:06.585763 10809 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:03:06.586848 10809 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:03:06.588048 10809 hybrid_clock.cc:648] HybridClock initialized: now 1786240986588037 us; error 38 us; skew 500 ppm
Aug 09 02:03:06 dist-test-slave-ss4n krb5kdc[10795](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240986, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:03:06.590952 10809 init.cc:377] Logged in from keytab as kudu/127.1.192.254@KRBTEST.COM (short username kudu)
I20260809 02:03:06.591902 10809 webserver.cc:460] Webserver started at http://127.1.192.254:41559/ using document root <none> and password file <none>
I20260809 02:03:06.592469 10809 fs_manager.cc:359] Metadata directory not provided
I20260809 02:03:06.592562 10809 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:03:06.592801 10809 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:03:06.594945 10809 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/master-0/data/instance:
uuid: "121837f9c6aa4210b18c06400933d754"
format_stamp: "Formatted at 2026-08-09 02:03:06 on dist-test-slave-ss4n"
server_key: "dc950dee7e2dba64c45822d1cc63ca2d"
server_key_iv: "869a0f32270f42c136fcd4b42741f906"
server_key_version: "encryptionkey@0"
I20260809 02:03:06.595546 10809 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/master-0/wal/instance:
uuid: "121837f9c6aa4210b18c06400933d754"
format_stamp: "Formatted at 2026-08-09 02:03:06 on dist-test-slave-ss4n"
server_key: "dc950dee7e2dba64c45822d1cc63ca2d"
server_key_iv: "869a0f32270f42c136fcd4b42741f906"
server_key_version: "encryptionkey@0"
I20260809 02:03:06.599517 10809 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 02:03:06.602082 10827 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:03:06.603339 10809 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 02:03:06.603484 10809 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/master-0/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/master-0/wal
uuid: "121837f9c6aa4210b18c06400933d754"
format_stamp: "Formatted at 2026-08-09 02:03:06 on dist-test-slave-ss4n"
server_key: "dc950dee7e2dba64c45822d1cc63ca2d"
server_key_iv: "869a0f32270f42c136fcd4b42741f906"
server_key_version: "encryptionkey@0"
I20260809 02:03:06.603605 10809 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:03:06.617942 10809 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:03:06.621842 10809 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:03:06.622092 10809 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:03:06.630554 10809 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.254:37079
I20260809 02:03:06.630585 10882 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.254:37079 every 8 connection(s)
I20260809 02:03:06.631592 10809 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
I20260809 02:03:06.633958 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 10809
I20260809 02:03:06.634114 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/master-0/wal/instance
I20260809 02:03:06.634420 1795 external_mini_cluster.cc:1428] Setting key f6bf27c45407904eee7208fbe649e007
I20260809 02:03:06.634644 10883 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:03:06.642124 10883 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 121837f9c6aa4210b18c06400933d754: Bootstrap starting.
Aug 09 02:03:06 dist-test-slave-ss4n krb5kdc[10795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240986, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.192.254@KRBTEST.COM
I20260809 02:03:06.644457 10883 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 121837f9c6aa4210b18c06400933d754: Neither blocks nor log segments found. Creating new log.
I20260809 02:03:06.645380 10883 log.cc:824] T 00000000000000000000000000000000 P 121837f9c6aa4210b18c06400933d754: Log is configured to *not* fsync() on all Append() calls
I20260809 02:03:06.647346 10883 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 121837f9c6aa4210b18c06400933d754: No bootstrap required, opened a new log
I20260809 02:03:06.649865 10887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:06.636051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45670 (local address 127.1.192.254:37079)
0809 02:03:06.636508 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0809 02:03:06.636528 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0809 02:03:06.636571 (+ 43us) server_negotiation.cc:407] Connection header received
0809 02:03:06.637523 (+ 952us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:03:06.637547 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:03:06.637972 (+ 425us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:03:06.638357 (+ 385us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:03:06.639135 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:06.640625 (+ 1490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:06.641536 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:06.642255 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:06.642417 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:06.644950 (+ 2533us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:03:06.644957 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:03:06.644972 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:03:06.645004 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:03:06.646852 (+ 1848us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:03:06.647327 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:03:06.647332 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:03:06.647337 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:03:06.647522 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:03:06.647821 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:03:06.647825 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:03:06.647827 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:03:06.648229 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:03:06.648425 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0809 02:03:06.648734 (+ 309us) server_negotiation.cc:299] Negotiation successful
0809 02:03:06.649052 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":136,"threads_started":1}
I20260809 02:03:06.651201 10883 raft_consensus.cc:348] T 00000000000000000000000000000000 P 121837f9c6aa4210b18c06400933d754 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "121837f9c6aa4210b18c06400933d754" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 37079 } }
I20260809 02:03:06.651476 10883 raft_consensus.cc:374] T 00000000000000000000000000000000 P 121837f9c6aa4210b18c06400933d754 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:03:06.651556 10883 raft_consensus.cc:729] T 00000000000000000000000000000000 P 121837f9c6aa4210b18c06400933d754 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 121837f9c6aa4210b18c06400933d754, State: Initialized, Role: FOLLOWER
I20260809 02:03:06.652035 10883 consensus_queue.cc:260] T 00000000000000000000000000000000 P 121837f9c6aa4210b18c06400933d754 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "121837f9c6aa4210b18c06400933d754" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 37079 } }
I20260809 02:03:06.652220 10883 raft_consensus.cc:388] T 00000000000000000000000000000000 P 121837f9c6aa4210b18c06400933d754 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:03:06.652300 10883 raft_consensus.cc:482] T 00000000000000000000000000000000 P 121837f9c6aa4210b18c06400933d754 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:03:06.652380 10883 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 121837f9c6aa4210b18c06400933d754 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:03:06.653290 10883 raft_consensus.cc:504] T 00000000000000000000000000000000 P 121837f9c6aa4210b18c06400933d754 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "121837f9c6aa4210b18c06400933d754" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 37079 } }
I20260809 02:03:06.653657 10883 leader_election.cc:302] T 00000000000000000000000000000000 P 121837f9c6aa4210b18c06400933d754 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 121837f9c6aa4210b18c06400933d754; no voters:
I20260809 02:03:06.653927 10883 leader_election.cc:288] T 00000000000000000000000000000000 P 121837f9c6aa4210b18c06400933d754 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:03:06.654086 10889 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 121837f9c6aa4210b18c06400933d754 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:03:06.654364 10889 raft_consensus.cc:686] T 00000000000000000000000000000000 P 121837f9c6aa4210b18c06400933d754 [term 1 LEADER]: Becoming Leader. State: Replica: 121837f9c6aa4210b18c06400933d754, State: Running, Role: LEADER
I20260809 02:03:06.654675 10889 consensus_queue.cc:237] T 00000000000000000000000000000000 P 121837f9c6aa4210b18c06400933d754 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "121837f9c6aa4210b18c06400933d754" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 37079 } }
I20260809 02:03:06.655239 10883 sys_catalog.cc:564] T 00000000000000000000000000000000 P 121837f9c6aa4210b18c06400933d754 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:03:06.656848 10891 sys_catalog.cc:455] T 00000000000000000000000000000000 P 121837f9c6aa4210b18c06400933d754 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 121837f9c6aa4210b18c06400933d754. Latest consensus state: current_term: 1 leader_uuid: "121837f9c6aa4210b18c06400933d754" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "121837f9c6aa4210b18c06400933d754" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 37079 } } }
I20260809 02:03:06.657001 10891 sys_catalog.cc:458] T 00000000000000000000000000000000 P 121837f9c6aa4210b18c06400933d754 [sys.catalog]: This master's current role is: LEADER
I20260809 02:03:06.657603 10901 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:03:06.657827 10890 sys_catalog.cc:455] T 00000000000000000000000000000000 P 121837f9c6aa4210b18c06400933d754 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "121837f9c6aa4210b18c06400933d754" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "121837f9c6aa4210b18c06400933d754" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 37079 } } }
I20260809 02:03:06.658172 10890 sys_catalog.cc:458] T 00000000000000000000000000000000 P 121837f9c6aa4210b18c06400933d754 [sys.catalog]: This master's current role is: LEADER
I20260809 02:03:06.660419 10901 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:03:06.666520 10901 catalog_manager.cc:1347] Generated new cluster ID: 6cb0b55cc9e34a5aaf6a4e9e0b4bfc3a
I20260809 02:03:06.666613 10901 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:03:06.681169 10901 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:03:06.682616 10901 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:03:06.691696 10901 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 121837f9c6aa4210b18c06400933d754: Generated new TSK 0
I20260809 02:03:06.692610 10901 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.192.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.192.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.192.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.193.keytab.
Entry for principal kudu/127.1.192.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.193.keytab.
I20260809 02:03:06.733330 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.192.193:0
--local_ip_for_outbound_sockets=127.1.192.193
--webserver_interface=127.1.192.193
--webserver_port=0
--tserver_master_addrs=127.1.192.254:37079
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.193.keytab
--principal=kudu/127.1.192.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:41989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:03:06.853204 10911 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:03:06.853587 10911 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:03:06.853663 10911 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:03:06.857995 10911 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:03:06.858076 10911 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:03:06.858141 10911 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.192.193
I20260809 02:03:06.863693 10911 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:41989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.193.keytab
--principal=kudu/127.1.192.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.192.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.192.193
--webserver_port=0
--tserver_master_addrs=127.1.192.254:37079
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10911
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.192.193
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:03:06.864775 10911 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:03:06.865811 10911 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 02:03:06.876276 10911 server_base.cc:1034] running on GCE node
W20260809 02:03:06.876223 10917 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:03:06.876223 10916 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:03:06.876236 10919 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:03:06.876992 10911 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:03:06.877746 10911 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:03:06.878933 10911 hybrid_clock.cc:648] HybridClock initialized: now 1786240986878899 us; error 46 us; skew 500 ppm
Aug 09 02:03:06 dist-test-slave-ss4n krb5kdc[10795](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240986, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:03:06.882814 10911 init.cc:377] Logged in from keytab as kudu/127.1.192.193@KRBTEST.COM (short username kudu)
I20260809 02:03:06.883694 10911 webserver.cc:460] Webserver started at http://127.1.192.193:37063/ using document root <none> and password file <none>
I20260809 02:03:06.884243 10911 fs_manager.cc:359] Metadata directory not provided
I20260809 02:03:06.884305 10911 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:03:06.884470 10911 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:03:06.886652 10911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/ts-0/data/instance:
uuid: "60c8706a4682415ea9bea2a1e4ec5f7c"
format_stamp: "Formatted at 2026-08-09 02:03:06 on dist-test-slave-ss4n"
server_key: "f14b75701bafd221124ef779a862ca8c"
server_key_iv: "9807a196bdfc8c85036d95813c74a8e0"
server_key_version: "encryptionkey@0"
I20260809 02:03:06.887212 10911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/ts-0/wal/instance:
uuid: "60c8706a4682415ea9bea2a1e4ec5f7c"
format_stamp: "Formatted at 2026-08-09 02:03:06 on dist-test-slave-ss4n"
server_key: "f14b75701bafd221124ef779a862ca8c"
server_key_iv: "9807a196bdfc8c85036d95813c74a8e0"
server_key_version: "encryptionkey@0"
I20260809 02:03:06.891359 10911 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260809 02:03:06.893998 10928 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:03:06.895357 10911 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 02:03:06.895520 10911 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/ts-0/wal
uuid: "60c8706a4682415ea9bea2a1e4ec5f7c"
format_stamp: "Formatted at 2026-08-09 02:03:06 on dist-test-slave-ss4n"
server_key: "f14b75701bafd221124ef779a862ca8c"
server_key_iv: "9807a196bdfc8c85036d95813c74a8e0"
server_key_version: "encryptionkey@0"
I20260809 02:03:06.895653 10911 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:03:06.904946 10911 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:03:06.908797 10911 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:03:06.909022 10911 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:03:06.909718 10911 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:03:06.910610 10911 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:03:06.910666 10911 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:03:06.910709 10911 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:03:06.910725 10911 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:03:06.920017 10911 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.193:44195
I20260809 02:03:06.920071 11054 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.193:44195 every 8 connection(s)
I20260809 02:03:06.921012 10911 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/ts-0/data/info.pb
Aug 09 02:03:06 dist-test-slave-ss4n krb5kdc[10795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240986, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.193@KRBTEST.COM for kudu/127.1.192.254@KRBTEST.COM
I20260809 02:03:06.931154 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 10911
I20260809 02:03:06.931282 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/ts-0/wal/instance
I20260809 02:03:06.931579 1795 external_mini_cluster.cc:1428] Setting key db615f5a3185f80b3864dd538248e0a6
I20260809 02:03:06.935719 10887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:06.923042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.193:50695 (local address 127.1.192.254:37079)
0809 02:03:06.923201 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0809 02:03:06.923206 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:03:06.924060 (+ 854us) server_negotiation.cc:407] Connection header received
0809 02:03:06.925440 (+ 1380us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:03:06.925443 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:03:06.925546 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:03:06.925655 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:03:06.926844 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:06.927386 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:06.928551 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:06.928801 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:06.928908 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:06.931836 (+ 2928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:03:06.931839 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:03:06.931841 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:03:06.931867 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:03:06.933600 (+ 1733us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:03:06.934124 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:03:06.934128 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:03:06.934130 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:03:06.934228 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:03:06.934589 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:03:06.934593 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:03:06.934594 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:03:06.934788 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:03:06.934897 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 02:03:06.935414 (+ 517us) server_negotiation.cc:299] Negotiation successful
0809 02:03:06.935547 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 02:03:06.936589 11057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:06.923428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:37079 (local address 127.1.192.193:50695)
0809 02:03:06.923905 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:06.923949 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 02:03:06.925222 (+ 1273us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:03:06.925834 (+ 612us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:03:06.925840 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:03:06.926212 (+ 372us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:03:06.926695 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:06.926708 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:06.927521 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:06.927530 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:06.928413 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:06.928422 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:06.928966 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:06.928970 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:06.929138 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:06.929804 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:03:06.929821 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:03:06.931605 (+ 1784us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:03:06.933721 (+ 2116us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:03:06.933727 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:03:06.933743 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:03:06.934018 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:03:06.934325 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:03:06.934329 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:03:06.934331 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:03:06.934485 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:03:06.934897 (+ 412us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:03:06.934905 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:03:06.935174 (+ 269us) client_negotiation.cc:769] Sending connection context
0809 02:03:06.935373 (+ 199us) client_negotiation.cc:240] Negotiation successful
0809 02:03:06.935736 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":132,"threads_started":1}
I20260809 02:03:06.937943 11055 heartbeater.cc:347] Connected to a master server at 127.1.192.254:37079
I20260809 02:03:06.938262 11055 heartbeater.cc:464] Registering TS with master...
I20260809 02:03:06.938936 11055 heartbeater.cc:511] Master 127.1.192.254:37079 requested a full tablet report, sending...
I20260809 02:03:06.940678 10844 ts_manager.cc:194] Registered new tserver with Master: 60c8706a4682415ea9bea2a1e4ec5f7c (127.1.192.193:44195)
I20260809 02:03:06.941916 10844 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.192.193@KRBTEST.COM'} at 127.1.192.193:50695
WARNING: no policy specified for kudu/127.1.192.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.192.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.192.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.194.keytab.
Entry for principal kudu/127.1.192.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.194.keytab.
I20260809 02:03:06.966928 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.192.194:0
--local_ip_for_outbound_sockets=127.1.192.194
--webserver_interface=127.1.192.194
--webserver_port=0
--tserver_master_addrs=127.1.192.254:37079
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.194.keytab
--principal=kudu/127.1.192.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:41989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:03:07.085733 11060 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:03:07.086009 11060 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:03:07.086054 11060 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:03:07.090025 11060 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:03:07.090085 11060 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:03:07.090147 11060 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.192.194
I20260809 02:03:07.095559 11060 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:41989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.194.keytab
--principal=kudu/127.1.192.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.192.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.192.194
--webserver_port=0
--tserver_master_addrs=127.1.192.254:37079
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11060
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.192.194
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:03:07.096634 11060 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:03:07.097647 11060 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:03:07.107401 11068 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:03:07.107522 11065 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:03:07.107380 11066 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:03:07.107889 11060 server_base.cc:1034] running on GCE node
I20260809 02:03:07.108321 11060 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:03:07.109014 11060 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:03:07.110213 11060 hybrid_clock.cc:648] HybridClock initialized: now 1786240987110187 us; error 39 us; skew 500 ppm
Aug 09 02:03:07 dist-test-slave-ss4n krb5kdc[10795](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240987, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:03:07.113148 11060 init.cc:377] Logged in from keytab as kudu/127.1.192.194@KRBTEST.COM (short username kudu)
I20260809 02:03:07.114167 11060 webserver.cc:460] Webserver started at http://127.1.192.194:45017/ using document root <none> and password file <none>
I20260809 02:03:07.114742 11060 fs_manager.cc:359] Metadata directory not provided
I20260809 02:03:07.114804 11060 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:03:07.114979 11060 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:03:07.116981 11060 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/ts-1/data/instance:
uuid: "965ef3e3ff0341cf842473b383e88351"
format_stamp: "Formatted at 2026-08-09 02:03:07 on dist-test-slave-ss4n"
server_key: "152143519982ce5a947278c420b91aca"
server_key_iv: "2823f1bf40d16a0a204b35fba5b3e8f1"
server_key_version: "encryptionkey@0"
I20260809 02:03:07.117576 11060 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/ts-1/wal/instance:
uuid: "965ef3e3ff0341cf842473b383e88351"
format_stamp: "Formatted at 2026-08-09 02:03:07 on dist-test-slave-ss4n"
server_key: "152143519982ce5a947278c420b91aca"
server_key_iv: "2823f1bf40d16a0a204b35fba5b3e8f1"
server_key_version: "encryptionkey@0"
I20260809 02:03:07.121248 11060 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260809 02:03:07.123886 11076 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:03:07.125300 11060 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 02:03:07.125409 11060 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/ts-1/wal
uuid: "965ef3e3ff0341cf842473b383e88351"
format_stamp: "Formatted at 2026-08-09 02:03:07 on dist-test-slave-ss4n"
server_key: "152143519982ce5a947278c420b91aca"
server_key_iv: "2823f1bf40d16a0a204b35fba5b3e8f1"
server_key_version: "encryptionkey@0"
I20260809 02:03:07.125676 11060 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:03:07.141669 11060 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:03:07.145264 11060 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:03:07.145452 11060 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:03:07.146178 11060 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:03:07.147099 11060 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:03:07.147161 11060 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:03:07.147205 11060 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:03:07.147231 11060 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:03:07.157678 11060 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.194:42499
I20260809 02:03:07.157764 11198 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.194:42499 every 8 connection(s)
I20260809 02:03:07.158710 11060 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/ts-1/data/info.pb
I20260809 02:03:07.165305 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 11060
I20260809 02:03:07.165462 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/ts-1/wal/instance
I20260809 02:03:07.165877 1795 external_mini_cluster.cc:1428] Setting key 3f0b697bb3a8e470be5852ee0a9330e0
Aug 09 02:03:07 dist-test-slave-ss4n krb5kdc[10795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240987, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.194@KRBTEST.COM for kudu/127.1.192.254@KRBTEST.COM
I20260809 02:03:07.174832 10887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:07.160644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.194:43527 (local address 127.1.192.254:37079)
0809 02:03:07.160817 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0809 02:03:07.160821 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:03:07.161790 (+ 969us) server_negotiation.cc:407] Connection header received
0809 02:03:07.163239 (+ 1449us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:03:07.163243 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:03:07.163310 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:03:07.163397 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:03:07.164672 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:07.165434 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:07.167093 (+ 1659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:07.167338 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:07.167515 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:07.170384 (+ 2869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:03:07.170389 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:03:07.170394 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:03:07.170427 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:03:07.172174 (+ 1747us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:03:07.172914 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:03:07.172933 (+ 19us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:03:07.172934 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:03:07.173031 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:03:07.173475 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:03:07.173518 (+ 43us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:03:07.173523 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:03:07.173739 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:03:07.173832 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 02:03:07.174587 (+ 755us) server_negotiation.cc:299] Negotiation successful
0809 02:03:07.174708 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 02:03:07.176097 11201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:07.161001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:37079 (local address 127.1.192.194:43527)
0809 02:03:07.161631 (+ 630us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:07.161679 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 02:03:07.163015 (+ 1336us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:03:07.163573 (+ 558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:03:07.163581 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:03:07.164009 (+ 428us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:03:07.164510 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:07.164524 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:07.165649 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:07.165653 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:07.166954 (+ 1301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:07.166965 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:07.167563 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:07.167567 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:07.167763 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:07.168450 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:03:07.168469 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:03:07.170203 (+ 1734us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:03:07.172303 (+ 2100us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:03:07.172313 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:03:07.172335 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:03:07.172739 (+ 404us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:03:07.173146 (+ 407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:03:07.173152 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:03:07.173154 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:03:07.173357 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:03:07.173859 (+ 502us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:03:07.173869 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:03:07.174263 (+ 394us) client_negotiation.cc:769] Sending connection context
0809 02:03:07.174522 (+ 259us) client_negotiation.cc:240] Negotiation successful
0809 02:03:07.174927 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":116,"threads_started":1}
I20260809 02:03:07.177414 11199 heartbeater.cc:347] Connected to a master server at 127.1.192.254:37079
WARNING: no policy specified for kudu/127.1.192.195@KRBTEST.COM; defaulting to no policy
I20260809 02:03:07.177700 11199 heartbeater.cc:464] Registering TS with master...
I20260809 02:03:07.178228 11199 heartbeater.cc:511] Master 127.1.192.254:37079 requested a full tablet report, sending...
I20260809 02:03:07.179415 10844 ts_manager.cc:194] Registered new tserver with Master: 965ef3e3ff0341cf842473b383e88351 (127.1.192.194:42499)
I20260809 02:03:07.180065 10844 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.192.194@KRBTEST.COM'} at 127.1.192.194:43527
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.192.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.192.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.195.keytab.
Entry for principal kudu/127.1.192.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.195.keytab.
I20260809 02:03:07.200506 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.192.195:0
--local_ip_for_outbound_sockets=127.1.192.195
--webserver_interface=127.1.192.195
--webserver_port=0
--tserver_master_addrs=127.1.192.254:37079
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.195.keytab
--principal=kudu/127.1.192.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:41989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:03:07.316259 11205 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:03:07.316593 11205 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:03:07.316673 11205 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:03:07.320779 11205 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:03:07.320891 11205 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:03:07.320988 11205 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.192.195
I20260809 02:03:07.326545 11205 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:41989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.195.keytab
--principal=kudu/127.1.192.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.192.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.192.195
--webserver_port=0
--tserver_master_addrs=127.1.192.254:37079
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11205
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.192.195
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:03:07.327700 11205 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:03:07.328822 11205 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:03:07.338795 11210 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:03:07.338795 11211 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:03:07.338842 11213 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:03:07.339115 11205 server_base.cc:1034] running on GCE node
I20260809 02:03:07.339529 11205 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:03:07.340224 11205 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:03:07.341434 11205 hybrid_clock.cc:648] HybridClock initialized: now 1786240987341402 us; error 43 us; skew 500 ppm
Aug 09 02:03:07 dist-test-slave-ss4n krb5kdc[10795](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240987, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:03:07.344791 11205 init.cc:377] Logged in from keytab as kudu/127.1.192.195@KRBTEST.COM (short username kudu)
I20260809 02:03:07.345928 11205 webserver.cc:460] Webserver started at http://127.1.192.195:42421/ using document root <none> and password file <none>
I20260809 02:03:07.346554 11205 fs_manager.cc:359] Metadata directory not provided
I20260809 02:03:07.346659 11205 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:03:07.346918 11205 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:03:07.349167 11205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/ts-2/data/instance:
uuid: "826b7f0d4919451fad7d34a92c6bf380"
format_stamp: "Formatted at 2026-08-09 02:03:07 on dist-test-slave-ss4n"
server_key: "207dc202434e9de583301f0e804cafac"
server_key_iv: "50751105a8a838be09e45e2d6725eb51"
server_key_version: "encryptionkey@0"
I20260809 02:03:07.349815 11205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/ts-2/wal/instance:
uuid: "826b7f0d4919451fad7d34a92c6bf380"
format_stamp: "Formatted at 2026-08-09 02:03:07 on dist-test-slave-ss4n"
server_key: "207dc202434e9de583301f0e804cafac"
server_key_iv: "50751105a8a838be09e45e2d6725eb51"
server_key_version: "encryptionkey@0"
I20260809 02:03:07.354008 11205 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260809 02:03:07.356907 11221 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:03:07.358448 11205 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 02:03:07.358662 11205 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/ts-2/wal
uuid: "826b7f0d4919451fad7d34a92c6bf380"
format_stamp: "Formatted at 2026-08-09 02:03:07 on dist-test-slave-ss4n"
server_key: "207dc202434e9de583301f0e804cafac"
server_key_iv: "50751105a8a838be09e45e2d6725eb51"
server_key_version: "encryptionkey@0"
I20260809 02:03:07.358790 11205 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:03:07.370162 11205 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:03:07.373747 11205 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:03:07.373965 11205 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:03:07.374588 11205 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:03:07.375442 11205 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:03:07.375524 11205 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:03:07.375597 11205 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:03:07.375649 11205 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:03:07.386205 11205 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.195:34029
I20260809 02:03:07.386210 11342 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.195:34029 every 8 connection(s)
I20260809 02:03:07.387143 11205 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/ts-2/data/info.pb
I20260809 02:03:07.388314 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 11205
I20260809 02:03:07.388458 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/ts-2/wal/instance
I20260809 02:03:07.388790 1795 external_mini_cluster.cc:1428] Setting key 0a57e8286964b7cfa91a3524aa668586
Aug 09 02:03:07 dist-test-slave-ss4n krb5kdc[10795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240987, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.195@KRBTEST.COM for kudu/127.1.192.254@KRBTEST.COM
I20260809 02:03:07.401903 10887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:07.389188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.195:47911 (local address 127.1.192.254:37079)
0809 02:03:07.389366 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0809 02:03:07.389371 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:03:07.390441 (+ 1070us) server_negotiation.cc:407] Connection header received
0809 02:03:07.391971 (+ 1530us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:03:07.391974 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:03:07.392020 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:03:07.392099 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:03:07.393284 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:07.393923 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:07.395089 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:07.395299 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:07.395381 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:07.398065 (+ 2684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:03:07.398068 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:03:07.398071 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:03:07.398098 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:03:07.399508 (+ 1410us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:03:07.400174 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:03:07.400177 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:03:07.400178 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:03:07.400266 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:03:07.400675 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:03:07.400678 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:03:07.400679 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:03:07.400863 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:03:07.400946 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 02:03:07.401583 (+ 637us) server_negotiation.cc:299] Negotiation successful
0809 02:03:07.401737 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 02:03:07.402909 11345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:07.389738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:37079 (local address 127.1.192.195:47911)
0809 02:03:07.390303 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:07.390347 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 02:03:07.391755 (+ 1408us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:03:07.392237 (+ 482us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:03:07.392243 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:03:07.392605 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:03:07.393147 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:07.393161 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:07.394045 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:07.394049 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:07.394974 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:07.394983 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:07.395431 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:07.395434 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:07.395615 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:07.396185 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:03:07.396201 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:03:07.397877 (+ 1676us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:03:07.399709 (+ 1832us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:03:07.399714 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:03:07.399727 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:03:07.400048 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:03:07.400358 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:03:07.400360 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:03:07.400361 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:03:07.400576 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:03:07.400963 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:03:07.400968 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:03:07.401208 (+ 240us) client_negotiation.cc:769] Sending connection context
0809 02:03:07.401374 (+ 166us) client_negotiation.cc:240] Negotiation successful
0809 02:03:07.401772 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":155,"threads_started":1}
I20260809 02:03:07.404340 11343 heartbeater.cc:347] Connected to a master server at 127.1.192.254:37079
I20260809 02:03:07.404618 11343 heartbeater.cc:464] Registering TS with master...
I20260809 02:03:07.405138 11343 heartbeater.cc:511] Master 127.1.192.254:37079 requested a full tablet report, sending...
I20260809 02:03:07.406347 10844 ts_manager.cc:194] Registered new tserver with Master: 826b7f0d4919451fad7d34a92c6bf380 (127.1.192.195:34029)
I20260809 02:03:07.406965 10844 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.192.195@KRBTEST.COM'} at 127.1.192.195:47911
I20260809 02:03:07.415768 1795 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.1.192.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.192.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.192.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/krb5kdc/oryx_127.1.192.193.keytab.
Entry for principal oryx/127.1.192.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/krb5kdc/oryx_127.1.192.193.keytab.
WARNING: no policy specified for oryx/127.1.192.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.192.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.192.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/krb5kdc/oryx_127.1.192.194.keytab.
Entry for principal oryx/127.1.192.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/krb5kdc/oryx_127.1.192.194.keytab.
WARNING: no policy specified for oryx/127.1.192.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.192.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.192.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/krb5kdc/oryx_127.1.192.195.keytab.
Entry for principal oryx/127.1.192.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/krb5kdc/oryx_127.1.192.195.keytab.
I20260809 02:03:07.504963 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 10911
I20260809 02:03:07.512065 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 11060
I20260809 02:03:07.519819 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 11205
I20260809 02:03:07.529523 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 10809
I20260809 02:03:07.538194 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.192.254:37079
--webserver_interface=127.1.192.254
--webserver_port=41559
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab
--principal=kudu/127.1.192.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:41989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.192.254:37079
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:03:07.654984 11353 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:03:07.655274 11353 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:03:07.655316 11353 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:03:07.659301 11353 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:03:07.659365 11353 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:03:07.659385 11353 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:03:07.659403 11353 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:03:07.659418 11353 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:03:07.664867 11353 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:41989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.192.254:37079
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab
--principal=kudu/127.1.192.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.192.254:37079
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.192.254
--webserver_port=41559
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11353
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:03:07.666018 11353 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:03:07.667037 11353 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:03:07.676364 11359 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:03:07.676388 11363 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:03:07.676376 11360 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:03:07.676440 11353 server_base.cc:1034] running on GCE node
I20260809 02:03:07.677224 11353 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:03:07.678500 11353 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:03:07.679709 11353 hybrid_clock.cc:648] HybridClock initialized: now 1786240987679695 us; error 29 us; skew 500 ppm
Aug 09 02:03:07 dist-test-slave-ss4n krb5kdc[10795](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240987, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:03:07.682667 11353 init.cc:377] Logged in from keytab as kudu/127.1.192.254@KRBTEST.COM (short username kudu)
I20260809 02:03:07.683575 11353 webserver.cc:460] Webserver started at http://127.1.192.254:41559/ using document root <none> and password file <none>
I20260809 02:03:07.684093 11353 fs_manager.cc:359] Metadata directory not provided
I20260809 02:03:07.684166 11353 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:03:07.688164 11353 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.000s
I20260809 02:03:07.690399 11371 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:03:07.691685 11353 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 02:03:07.691826 11353 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/master-0/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/master-0/wal
uuid: "121837f9c6aa4210b18c06400933d754"
format_stamp: "Formatted at 2026-08-09 02:03:06 on dist-test-slave-ss4n"
server_key: "dc950dee7e2dba64c45822d1cc63ca2d"
server_key_iv: "869a0f32270f42c136fcd4b42741f906"
server_key_version: "encryptionkey@0"
I20260809 02:03:07.692395 11353 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:03:07.713981 11353 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:03:07.717339 11353 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:03:07.717603 11353 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:03:07.725332 11353 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.254:37079
I20260809 02:03:07.725363 11430 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.254:37079 every 8 connection(s)
I20260809 02:03:07.726526 11353 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
I20260809 02:03:07.731004 11431 sys_catalog.cc:263] Verifying existing consensus state
I20260809 02:03:07.732728 11431 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 121837f9c6aa4210b18c06400933d754: Bootstrap starting.
I20260809 02:03:07.735998 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 11353
I20260809 02:03:07.736522 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.192.193:44195
--local_ip_for_outbound_sockets=127.1.192.193
--tserver_master_addrs=127.1.192.254:37079
--webserver_port=37063
--webserver_interface=127.1.192.193
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.193.keytab
--principal=kudu/127.1.192.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:41989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.1.192.193
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/krb5kdc/oryx_127.1.192.193.keytab with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 02:03:07.751256 11431 log.cc:824] T 00000000000000000000000000000000 P 121837f9c6aa4210b18c06400933d754: Log is configured to *not* fsync() on all Append() calls
I20260809 02:03:07.758222 11431 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 121837f9c6aa4210b18c06400933d754: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 02:03:07.758638 11431 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 121837f9c6aa4210b18c06400933d754: Bootstrap complete.
I20260809 02:03:07.763556 11431 raft_consensus.cc:348] T 00000000000000000000000000000000 P 121837f9c6aa4210b18c06400933d754 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "121837f9c6aa4210b18c06400933d754" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 37079 } }
I20260809 02:03:07.764001 11431 raft_consensus.cc:729] T 00000000000000000000000000000000 P 121837f9c6aa4210b18c06400933d754 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 121837f9c6aa4210b18c06400933d754, State: Initialized, Role: FOLLOWER
I20260809 02:03:07.764571 11431 consensus_queue.cc:260] T 00000000000000000000000000000000 P 121837f9c6aa4210b18c06400933d754 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "121837f9c6aa4210b18c06400933d754" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 37079 } }
I20260809 02:03:07.764698 11431 raft_consensus.cc:388] T 00000000000000000000000000000000 P 121837f9c6aa4210b18c06400933d754 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:03:07.764782 11431 raft_consensus.cc:482] T 00000000000000000000000000000000 P 121837f9c6aa4210b18c06400933d754 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:03:07.764880 11431 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 121837f9c6aa4210b18c06400933d754 [term 1 FOLLOWER]: Advancing to term 2
I20260809 02:03:07.766980 11431 raft_consensus.cc:504] T 00000000000000000000000000000000 P 121837f9c6aa4210b18c06400933d754 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "121837f9c6aa4210b18c06400933d754" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 37079 } }
I20260809 02:03:07.767329 11431 leader_election.cc:302] T 00000000000000000000000000000000 P 121837f9c6aa4210b18c06400933d754 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 121837f9c6aa4210b18c06400933d754; no voters:
I20260809 02:03:07.767676 11431 leader_election.cc:288] T 00000000000000000000000000000000 P 121837f9c6aa4210b18c06400933d754 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 02:03:07.767792 11435 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 121837f9c6aa4210b18c06400933d754 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 02:03:07.768014 11435 raft_consensus.cc:686] T 00000000000000000000000000000000 P 121837f9c6aa4210b18c06400933d754 [term 2 LEADER]: Becoming Leader. State: Replica: 121837f9c6aa4210b18c06400933d754, State: Running, Role: LEADER
I20260809 02:03:07.768322 11435 consensus_queue.cc:237] T 00000000000000000000000000000000 P 121837f9c6aa4210b18c06400933d754 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "121837f9c6aa4210b18c06400933d754" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 37079 } }
I20260809 02:03:07.768940 11431 sys_catalog.cc:564] T 00000000000000000000000000000000 P 121837f9c6aa4210b18c06400933d754 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:03:07.769292 11435 sys_catalog.cc:455] T 00000000000000000000000000000000 P 121837f9c6aa4210b18c06400933d754 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 121837f9c6aa4210b18c06400933d754. Latest consensus state: current_term: 2 leader_uuid: "121837f9c6aa4210b18c06400933d754" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "121837f9c6aa4210b18c06400933d754" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 37079 } } }
I20260809 02:03:07.769412 11435 sys_catalog.cc:458] T 00000000000000000000000000000000 P 121837f9c6aa4210b18c06400933d754 [sys.catalog]: This master's current role is: LEADER
I20260809 02:03:07.770061 11436 sys_catalog.cc:455] T 00000000000000000000000000000000 P 121837f9c6aa4210b18c06400933d754 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "121837f9c6aa4210b18c06400933d754" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "121837f9c6aa4210b18c06400933d754" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 37079 } } }
I20260809 02:03:07.770175 11436 sys_catalog.cc:458] T 00000000000000000000000000000000 P 121837f9c6aa4210b18c06400933d754 [sys.catalog]: This master's current role is: LEADER
I20260809 02:03:07.774535 11452 catalog_manager.cc:1259] Loaded cluster ID: 6cb0b55cc9e34a5aaf6a4e9e0b4bfc3a
I20260809 02:03:07.774606 11452 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 121837f9c6aa4210b18c06400933d754: loading cluster ID for follower catalog manager: success
I20260809 02:03:07.777231 11452 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 121837f9c6aa4210b18c06400933d754: acquiring CA information for follower catalog manager: success
I20260809 02:03:07.778748 11452 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 121837f9c6aa4210b18c06400933d754: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 02:03:07.778913 11439 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:03:07.779932 11439 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:03:07.780416 11439 catalog_manager.cc:1259] Loaded cluster ID: 6cb0b55cc9e34a5aaf6a4e9e0b4bfc3a
I20260809 02:03:07.780478 11439 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:03:07.781208 11439 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:03:07.781733 11439 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 121837f9c6aa4210b18c06400933d754: Loaded TSK: 0
I20260809 02:03:07.782550 11439 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 02:03:07.863286 11433 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:03:07.863571 11433 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:03:07.863619 11433 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:03:07.867913 11433 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:03:07.867988 11433 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:03:07.868052 11433 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.192.193
I20260809 02:03:07.874114 11433 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:41989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/krb5kdc/oryx_127.1.192.193.keytab
--principal=oryx/127.1.192.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.192.193:44195
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.192.193
--webserver_port=37063
--tserver_master_addrs=127.1.192.254:37079
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11433
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.192.193
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:03:07.875196 11433 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:03:07.876202 11433 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:03:07.886878 11460 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:03:07.886893 11459 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:03:07.887353 11462 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:03:07.887696 11433 server_base.cc:1034] running on GCE node
I20260809 02:03:07.888128 11433 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:03:07.888803 11433 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:03:07.889976 11433 hybrid_clock.cc:648] HybridClock initialized: now 1786240987889934 us; error 58 us; skew 500 ppm
Aug 09 02:03:07 dist-test-slave-ss4n krb5kdc[10795](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240987, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.192.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:03:07.892833 11433 init.cc:377] Logged in from keytab as oryx/127.1.192.193@KRBTEST.COM (short username oryx)
I20260809 02:03:07.893976 11433 webserver.cc:460] Webserver started at http://127.1.192.193:37063/ using document root <none> and password file <none>
I20260809 02:03:07.894567 11433 fs_manager.cc:359] Metadata directory not provided
I20260809 02:03:07.894668 11433 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:03:07.898830 11433 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 02:03:07.901031 11470 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:03:07.902366 11433 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 02:03:07.902504 11433 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/ts-0/wal
uuid: "60c8706a4682415ea9bea2a1e4ec5f7c"
format_stamp: "Formatted at 2026-08-09 02:03:06 on dist-test-slave-ss4n"
server_key: "f14b75701bafd221124ef779a862ca8c"
server_key_iv: "9807a196bdfc8c85036d95813c74a8e0"
server_key_version: "encryptionkey@0"
I20260809 02:03:07.903051 11433 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:03:07.941004 11433 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:03:07.945322 11433 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:03:07.945642 11433 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:03:07.946409 11433 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:03:07.947441 11433 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:03:07.947500 11433 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:03:07.947583 11433 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:03:07.947626 11433 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:03:07.958056 11433 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.193:44195
I20260809 02:03:07.958082 11598 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.193:44195 every 8 connection(s)
I20260809 02:03:07.959136 11433 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/ts-0/data/info.pb
I20260809 02:03:07.965152 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 11433
I20260809 02:03:07.965694 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.192.194:42499
--local_ip_for_outbound_sockets=127.1.192.194
--tserver_master_addrs=127.1.192.254:37079
--webserver_port=45017
--webserver_interface=127.1.192.194
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.194.keytab
--principal=kudu/127.1.192.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:41989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.1.192.194
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/krb5kdc/oryx_127.1.192.194.keytab with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 02:03:07 dist-test-slave-ss4n krb5kdc[10795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.192.193@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:03:07 dist-test-slave-ss4n krb5kdc[10795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.192.193@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:03:07.975545 11602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:07.961656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:37079 (local address 127.1.192.193:60987)
0809 02:03:07.962214 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:07.962263 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 02:03:07.963529 (+ 1266us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:03:07.964653 (+ 1124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:03:07.964660 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:03:07.965073 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:03:07.965689 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:07.965706 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:07.967645 (+ 1939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:07.967651 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:07.969161 (+ 1510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:07.969175 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:07.970204 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:07.970210 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:07.970438 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:07.971235 (+ 797us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:03:07.971262 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:03:07.974439 (+ 3177us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.192.254:37079: Server oryx/127.1.192.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":146,"threads_started":1}
W20260809 02:03:07.975533 11601 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:03:07.961545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.193:60987 (local address 127.1.192.254:37079)
0809 02:03:07.962084 (+ 539us) server_negotiation.cc:206] Beginning negotiation
0809 02:03:07.962105 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0809 02:03:07.962390 (+ 285us) server_negotiation.cc:407] Connection header received
0809 02:03:07.963909 (+ 1519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:03:07.963920 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:03:07.964237 (+ 317us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:03:07.964499 (+ 262us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:03:07.965910 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:07.967450 (+ 1540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:07.969311 (+ 1861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:07.970010 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:07.970153 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:07.974717 (+ 4564us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.192.193:60987: BlockingRecv error: recv got EOF from 127.1.192.193:60987 (error 108)
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":149,"threads_started":1}
W20260809 02:03:07.976429 11599 heartbeater.cc:644] Failed to heartbeat to 127.1.192.254:37079 (0 consecutive failures): Not authorized: Failed to ping master at 127.1.192.254:37079: Client connection negotiation failed: client connection to 127.1.192.254:37079: Server oryx/127.1.192.254@KRBTEST.COM not found in Kerberos database
Aug 09 02:03:07 dist-test-slave-ss4n krb5kdc[10795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.192.193@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:03:07 dist-test-slave-ss4n krb5kdc[10795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.192.193@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:03:07.982722 11602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:07.977000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:37079 (local address 127.1.192.193:35845)
0809 02:03:07.977108 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:07.977126 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 02:03:07.977221 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:03:07.977576 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:03:07.977580 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:03:07.977770 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:03:07.977872 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:07.977878 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:07.978723 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:07.978730 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:07.979343 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:07.979352 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:07.979865 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:07.979869 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:07.979981 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:07.980535 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:03:07.980548 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:03:07.982561 (+ 2013us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.192.254:37079: Server oryx/127.1.192.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":34}
W20260809 02:03:07.982824 11601 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:03:07.977129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.193:35845 (local address 127.1.192.254:37079)
0809 02:03:07.977282 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0809 02:03:07.977287 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:03:07.977302 (+ 15us) server_negotiation.cc:407] Connection header received
0809 02:03:07.977353 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:03:07.977355 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:03:07.977398 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:03:07.977560 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:03:07.978006 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:07.978601 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:07.979492 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:07.979735 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:07.979850 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:07.982698 (+ 2848us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.192.193:35845: BlockingRecv error: recv got EOF from 127.1.192.193:35845 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 09 02:03:07 dist-test-slave-ss4n krb5kdc[10795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.192.193@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:03:07 dist-test-slave-ss4n krb5kdc[10795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.192.193@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:03:07.988624 11602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:07.983520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:37079 (local address 127.1.192.193:37811)
0809 02:03:07.983648 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:07.983662 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 02:03:07.983740 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:03:07.983960 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:03:07.983964 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:03:07.984136 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:03:07.984220 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:07.984226 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:07.985047 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:07.985051 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:07.985636 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:07.985646 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:07.986118 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:07.986122 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:07.986210 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:07.986701 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:03:07.986713 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:03:07.988486 (+ 1773us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.192.254:37079: Server oryx/127.1.192.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":55}
W20260809 02:03:07.988752 11601 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:03:07.983636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.193:37811 (local address 127.1.192.254:37079)
0809 02:03:07.983746 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0809 02:03:07.983750 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:03:07.983763 (+ 13us) server_negotiation.cc:407] Connection header received
0809 02:03:07.983817 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:03:07.983819 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:03:07.983856 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:03:07.983923 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:03:07.984353 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:07.984920 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:07.985753 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:07.985972 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:07.986102 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:07.988623 (+ 2521us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.192.193:37811: BlockingRecv error: recv got EOF from 127.1.192.193:37811 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
W20260809 02:03:07.988859 11599 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 02:03:08.084030 11603 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:03:08.084290 11603 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:03:08.084333 11603 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:03:08.088593 11603 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:03:08.088663 11603 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:03:08.088724 11603 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.192.194
I20260809 02:03:08.094269 11603 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:41989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/krb5kdc/oryx_127.1.192.194.keytab
--principal=oryx/127.1.192.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.192.194:42499
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.192.194
--webserver_port=45017
--tserver_master_addrs=127.1.192.254:37079
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11603
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.192.194
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:03:08.095275 11603 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:03:08.096570 11603 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:03:08.106865 11609 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:03:08.106948 11608 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:03:08.106873 11611 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:03:08.107642 11603 server_base.cc:1034] running on GCE node
I20260809 02:03:08.108110 11603 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:03:08.108806 11603 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:03:08.109973 11603 hybrid_clock.cc:648] HybridClock initialized: now 1786240988109939 us; error 47 us; skew 500 ppm
Aug 09 02:03:08 dist-test-slave-ss4n krb5kdc[10795](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240988, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.192.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:03:08.113144 11603 init.cc:377] Logged in from keytab as oryx/127.1.192.194@KRBTEST.COM (short username oryx)
I20260809 02:03:08.114154 11603 webserver.cc:460] Webserver started at http://127.1.192.194:45017/ using document root <none> and password file <none>
I20260809 02:03:08.114719 11603 fs_manager.cc:359] Metadata directory not provided
I20260809 02:03:08.114800 11603 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:03:08.118960 11603 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 02:03:08.121277 11618 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:03:08.122746 11603 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 02:03:08.122886 11603 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/ts-1/wal
uuid: "965ef3e3ff0341cf842473b383e88351"
format_stamp: "Formatted at 2026-08-09 02:03:07 on dist-test-slave-ss4n"
server_key: "152143519982ce5a947278c420b91aca"
server_key_iv: "2823f1bf40d16a0a204b35fba5b3e8f1"
server_key_version: "encryptionkey@0"
I20260809 02:03:08.123454 11603 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:03:08.138886 11603 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:03:08.142768 11603 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:03:08.142984 11603 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:03:08.143673 11603 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:03:08.144589 11603 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:03:08.144667 11603 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:03:08.144745 11603 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:03:08.144799 11603 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:03:08.154724 11603 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.194:42499
I20260809 02:03:08.154762 11745 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.194:42499 every 8 connection(s)
I20260809 02:03:08.155920 11603 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/ts-1/data/info.pb
I20260809 02:03:08.164865 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 11603
I20260809 02:03:08.165539 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.192.195:34029
--local_ip_for_outbound_sockets=127.1.192.195
--tserver_master_addrs=127.1.192.254:37079
--webserver_port=42421
--webserver_interface=127.1.192.195
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.195.keytab
--principal=kudu/127.1.192.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:41989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.1.192.195
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/krb5kdc/oryx_127.1.192.195.keytab with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 02:03:08 dist-test-slave-ss4n krb5kdc[10795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.192.194@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:03:08 dist-test-slave-ss4n krb5kdc[10795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.192.194@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM, Server not found in Kerberos database
W20260809 02:03:08.169368 11601 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:03:08.157951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.194:38893 (local address 127.1.192.254:37079)
0809 02:03:08.158130 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0809 02:03:08.158134 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:03:08.159033 (+ 899us) server_negotiation.cc:407] Connection header received
0809 02:03:08.160347 (+ 1314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:03:08.160351 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:03:08.160408 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:03:08.160497 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:03:08.161933 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:08.162579 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:08.164207 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:08.164445 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:08.164588 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:08.169226 (+ 4638us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.192.194:38893: BlockingRecv error: recv got EOF from 127.1.192.194:38893 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
I20260809 02:03:08.170192 11749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:08.158388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:37079 (local address 127.1.192.194:38893)
0809 02:03:08.158857 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:08.158903 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 02:03:08.160116 (+ 1213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:03:08.160779 (+ 663us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:03:08.160786 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:03:08.161151 (+ 365us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:03:08.161771 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:08.161788 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:08.162820 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:08.162823 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:08.164058 (+ 1235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:08.164069 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:08.164668 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:08.164674 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:08.164924 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:08.165959 (+ 1035us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:03:08.165988 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:03:08.169022 (+ 3034us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.192.254:37079: Server oryx/127.1.192.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":125,"threads_started":1}
W20260809 02:03:08.170729 11747 heartbeater.cc:644] Failed to heartbeat to 127.1.192.254:37079 (0 consecutive failures): Not authorized: Failed to ping master at 127.1.192.254:37079: Client connection negotiation failed: client connection to 127.1.192.254:37079: Server oryx/127.1.192.254@KRBTEST.COM not found in Kerberos database
Aug 09 02:03:08 dist-test-slave-ss4n krb5kdc[10795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.192.194@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:03:08 dist-test-slave-ss4n krb5kdc[10795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.192.194@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:03:08.176649 11749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:08.171282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:37079 (local address 127.1.192.194:33473)
0809 02:03:08.171409 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:08.171425 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 02:03:08.171517 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:03:08.171871 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:03:08.171874 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:03:08.172049 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:03:08.172161 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:08.172166 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:08.172931 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:08.172935 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:08.173584 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:08.173594 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:08.174056 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:08.174061 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:08.174159 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:08.174699 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:03:08.174720 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:03:08.176493 (+ 1773us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.192.254:37079: Server oryx/127.1.192.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":38}
W20260809 02:03:08.176738 11601 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:03:08.171354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.194:33473 (local address 127.1.192.254:37079)
0809 02:03:08.171473 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0809 02:03:08.171478 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:03:08.171492 (+ 14us) server_negotiation.cc:407] Connection header received
0809 02:03:08.171709 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:03:08.171713 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:03:08.171767 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:03:08.171852 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:03:08.172278 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:08.172812 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:08.173703 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:08.173938 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:08.174058 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:08.176616 (+ 2558us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.192.194:33473: BlockingRecv error: recv got EOF from 127.1.192.194:33473 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 09 02:03:08 dist-test-slave-ss4n krb5kdc[10795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.192.194@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:03:08 dist-test-slave-ss4n krb5kdc[10795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.192.194@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:03:08.183393 11749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:08.177619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:37079 (local address 127.1.192.194:33641)
0809 02:03:08.177754 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:08.177770 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 02:03:08.177846 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:03:08.178150 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:03:08.178152 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:03:08.178276 (+ 124us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:03:08.178349 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:08.178354 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:08.179580 (+ 1226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:08.179583 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:08.180095 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:08.180103 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:08.180712 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:08.180714 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:08.180772 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:08.181322 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:03:08.181333 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:03:08.183207 (+ 1874us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.192.254:37079: Server oryx/127.1.192.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":55}
W20260809 02:03:08.183660 11747 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 02:03:08.183475 11601 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:03:08.177719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.194:33641 (local address 127.1.192.254:37079)
0809 02:03:08.177877 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0809 02:03:08.177884 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 02:03:08.177903 (+ 19us) server_negotiation.cc:407] Connection header received
0809 02:03:08.177954 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:03:08.177960 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:03:08.178023 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:03:08.178135 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:03:08.178474 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:08.179449 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:08.180209 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:08.180599 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:08.180706 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:08.183346 (+ 2640us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.192.194:33641: BlockingRecv error: recv got EOF from 127.1.192.194:33641 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260809 02:03:08.288230 11750 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:03:08.288504 11750 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:03:08.288549 11750 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:03:08.292716 11750 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:03:08.292790 11750 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:03:08.292888 11750 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.192.195
I20260809 02:03:08.299256 11750 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:41989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/krb5kdc/oryx_127.1.192.195.keytab
--principal=oryx/127.1.192.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.192.195:34029
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.192.195
--webserver_port=42421
--tserver_master_addrs=127.1.192.254:37079
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11750
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.192.195
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:03:08.300390 11750 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:03:08.301431 11750 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:03:08.311802 11757 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:03:08.311856 11756 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:03:08.311779 11759 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:03:08.312093 11750 server_base.cc:1034] running on GCE node
I20260809 02:03:08.312609 11750 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:03:08.313412 11750 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:03:08.314669 11750 hybrid_clock.cc:648] HybridClock initialized: now 1786240988314643 us; error 38 us; skew 500 ppm
Aug 09 02:03:08 dist-test-slave-ss4n krb5kdc[10795](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786240988, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.192.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:03:08.318012 11750 init.cc:377] Logged in from keytab as oryx/127.1.192.195@KRBTEST.COM (short username oryx)
I20260809 02:03:08.319016 11750 webserver.cc:460] Webserver started at http://127.1.192.195:42421/ using document root <none> and password file <none>
I20260809 02:03:08.319608 11750 fs_manager.cc:359] Metadata directory not provided
I20260809 02:03:08.319741 11750 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:03:08.323827 11750 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.004s
I20260809 02:03:08.326164 11767 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:03:08.327571 11750 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 02:03:08.327745 11750 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/ts-2/wal
uuid: "826b7f0d4919451fad7d34a92c6bf380"
format_stamp: "Formatted at 2026-08-09 02:03:07 on dist-test-slave-ss4n"
server_key: "207dc202434e9de583301f0e804cafac"
server_key_iv: "50751105a8a838be09e45e2d6725eb51"
server_key_version: "encryptionkey@0"
I20260809 02:03:08.328397 11750 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:03:08.348003 11750 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:03:08.351377 11750 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:03:08.351593 11750 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:03:08.352216 11750 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:03:08.353121 11750 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:03:08.353195 11750 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:03:08.353269 11750 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:03:08.353320 11750 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:03:08.363540 11750 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.195:34029
I20260809 02:03:08.363564 11891 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.195:34029 every 8 connection(s)
I20260809 02:03:08.364583 11750 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786240941968412-1795-0/minicluster-data/ts-2/data/info.pb
I20260809 02:03:08.374979 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 11750
Aug 09 02:03:08 dist-test-slave-ss4n krb5kdc[10795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.192.195@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:03:08 dist-test-slave-ss4n krb5kdc[10795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.192.195@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM, Server not found in Kerberos database
W20260809 02:03:08.377802 11601 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:03:08.366663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.195:59875 (local address 127.1.192.254:37079)
0809 02:03:08.366823 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0809 02:03:08.366827 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:03:08.367682 (+ 855us) server_negotiation.cc:407] Connection header received
0809 02:03:08.369241 (+ 1559us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:03:08.369245 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:03:08.369313 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:03:08.369443 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:03:08.370926 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:08.371557 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:08.373100 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:08.373362 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:08.373472 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:08.377603 (+ 4131us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.192.195:59875: BlockingRecv error: recv got EOF from 127.1.192.195:59875 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 02:03:08.378450 11894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:08.366947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:37079 (local address 127.1.192.195:59875)
0809 02:03:08.367487 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:08.367536 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 02:03:08.368991 (+ 1455us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:03:08.369732 (+ 741us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:03:08.369740 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:03:08.370170 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:03:08.370754 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:08.370769 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:08.371725 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:08.371728 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:08.372948 (+ 1220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:08.372961 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:08.373613 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:08.373617 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:08.373806 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:08.374574 (+ 768us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:03:08.374594 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:03:08.377402 (+ 2808us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.192.254:37079: Server oryx/127.1.192.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":313,"thread_start_us":125,"threads_started":1}
W20260809 02:03:08.379168 11892 heartbeater.cc:644] Failed to heartbeat to 127.1.192.254:37079 (0 consecutive failures): Not authorized: Failed to ping master at 127.1.192.254:37079: Client connection negotiation failed: client connection to 127.1.192.254:37079: Server oryx/127.1.192.254@KRBTEST.COM not found in Kerberos database
Aug 09 02:03:08 dist-test-slave-ss4n krb5kdc[10795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.192.195@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:03:08 dist-test-slave-ss4n krb5kdc[10795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.192.195@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:03:08.386325 11894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:08.379800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:37079 (local address 127.1.192.195:53461)
0809 02:03:08.380041 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:08.380054 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:03:08.380129 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:03:08.380354 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:03:08.380356 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:03:08.380504 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:03:08.380651 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:08.380658 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:08.381436 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:08.381443 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:08.382422 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:08.382432 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:08.383016 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:08.383021 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:08.383154 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:08.383616 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:03:08.383630 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:03:08.386146 (+ 2516us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.192.254:37079: Server oryx/127.1.192.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":164}
W20260809 02:03:08.386466 11601 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:03:08.379942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.195:53461 (local address 127.1.192.254:37079)
0809 02:03:08.380128 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0809 02:03:08.380132 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:03:08.380146 (+ 14us) server_negotiation.cc:407] Connection header received
0809 02:03:08.380196 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:03:08.380199 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:03:08.380245 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:03:08.380321 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:03:08.380774 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:08.381321 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:08.382531 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:08.382880 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:08.383013 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:08.386309 (+ 3296us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.192.195:53461: BlockingRecv error: recv got EOF from 127.1.192.195:53461 (error 108)
Metrics: {"server-negotiator.queue_time_us":91}
I20260809 02:03:08.388381 11896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:08.376304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45674 (local address 127.1.192.254:37079)
0809 02:03:08.376628 (+ 324us) server_negotiation.cc:206] Beginning negotiation
0809 02:03:08.376633 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:03:08.376793 (+ 160us) server_negotiation.cc:407] Connection header received
0809 02:03:08.376975 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:03:08.376977 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:03:08.377021 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:03:08.377115 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:03:08.378051 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:08.378612 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:08.380118 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:08.380513 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:08.380600 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:08.382460 (+ 1860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:03:08.382466 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:03:08.382528 (+ 62us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:03:08.382566 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:03:08.385148 (+ 2582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:03:08.385875 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:03:08.385883 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:03:08.385892 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:03:08.386097 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:03:08.386784 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:03:08.386788 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:03:08.386790 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:03:08.387343 (+ 553us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:03:08.387606 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0809 02:03:08.387948 (+ 342us) server_negotiation.cc:299] Negotiation successful
0809 02:03:08.388236 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":225,"thread_start_us":107,"threads_started":1}
Aug 09 02:03:08 dist-test-slave-ss4n krb5kdc[10795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.192.195@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:03:08 dist-test-slave-ss4n krb5kdc[10795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.192.195@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:03:08.392841 11894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:08.386984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:37079 (local address 127.1.192.195:51597)
0809 02:03:08.387237 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:08.387250 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:03:08.387361 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:03:08.387640 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:03:08.387643 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:03:08.387801 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:03:08.387874 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:08.387879 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:08.388788 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:08.388791 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:08.389365 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:08.389374 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:08.389884 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:08.389888 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:08.389988 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:08.390527 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:03:08.390543 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:03:08.392737 (+ 2194us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.192.254:37079: Server oryx/127.1.192.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":56}
W20260809 02:03:08.393064 11892 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 02:03:08.393265 11601 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:03:08.387064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.195:51597 (local address 127.1.192.254:37079)
0809 02:03:08.387210 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0809 02:03:08.387213 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:03:08.387309 (+ 96us) server_negotiation.cc:407] Connection header received
0809 02:03:08.387460 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:03:08.387463 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:03:08.387530 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:03:08.387615 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:03:08.387987 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:08.388668 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:08.389540 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:08.389764 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:08.389843 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:08.392871 (+ 3028us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.192.195:51597: BlockingRecv error: recv got EOF from 127.1.192.195:51597 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 09 02:03:08 dist-test-slave-ss4n krb5kdc[10795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.192.193@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:03:08 dist-test-slave-ss4n krb5kdc[10795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.192.193@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:03:08.995067 11897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:08.989524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:37079 (local address 127.1.192.193:46661)
0809 02:03:08.989869 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:08.989883 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 02:03:08.989980 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:03:08.990226 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:03:08.990228 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:03:08.990412 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:03:08.990509 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:08.990514 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:08.991263 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:08.991265 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:08.991898 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:08.991908 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:08.992359 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:08.992361 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:08.992422 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:08.992895 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:03:08.992906 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:03:08.994931 (+ 2025us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.192.254:37079: Server oryx/127.1.192.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":91,"threads_started":1}
W20260809 02:03:08.995359 11898 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:03:08.989585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.193:46661 (local address 127.1.192.254:37079)
0809 02:03:08.989850 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0809 02:03:08.989854 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:03:08.989950 (+ 96us) server_negotiation.cc:407] Connection header received
0809 02:03:08.990059 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:03:08.990062 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:03:08.990107 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:03:08.990191 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:03:08.990626 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:08.991159 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:08.992017 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:08.992258 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:08.992342 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:08.995093 (+ 2751us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.192.193:46661: BlockingRecv error: recv got EOF from 127.1.192.193:46661 (error 108)
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":89,"threads_started":1}
Aug 09 02:03:09 dist-test-slave-ss4n krb5kdc[10795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.192.194@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:03:09 dist-test-slave-ss4n krb5kdc[10795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.192.194@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:03:09.191226 11899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:09.184393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:37079 (local address 127.1.192.194:42787)
0809 02:03:09.184676 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:09.184695 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 02:03:09.184804 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:03:09.185217 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:03:09.185221 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:03:09.185589 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:03:09.185782 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:09.185796 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:09.186715 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:09.186719 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:09.187533 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:09.187544 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:09.188014 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:09.188017 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:09.188085 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:09.188705 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:03:09.188717 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:03:09.191021 (+ 2304us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.192.254:37079: Server oryx/127.1.192.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":157,"threads_started":1}
W20260809 02:03:09.191268 11898 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:03:09.184509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.194:42787 (local address 127.1.192.254:37079)
0809 02:03:09.184731 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0809 02:03:09.184734 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:03:09.184754 (+ 20us) server_negotiation.cc:407] Connection header received
0809 02:03:09.184928 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:03:09.184931 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:03:09.184986 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:03:09.185124 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:03:09.185929 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:09.186600 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:09.187655 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:09.187874 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:09.187956 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:09.191147 (+ 3191us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.192.194:42787: BlockingRecv error: recv got EOF from 127.1.192.194:42787 (error 108)
Metrics: {"server-negotiator.queue_time_us":131}
Aug 09 02:03:09 dist-test-slave-ss4n krb5kdc[10795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.192.195@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:03:09 dist-test-slave-ss4n krb5kdc[10795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.192.195@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:03:09.400409 11900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:09.393818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:37079 (local address 127.1.192.195:52539)
0809 02:03:09.394180 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:09.394195 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 02:03:09.394353 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:03:09.394658 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:03:09.394662 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:03:09.394883 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:03:09.394981 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:09.394988 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:09.395916 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:09.395922 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:09.396597 (+ 675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:09.396607 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:09.397174 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:09.397178 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:09.397284 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:09.397883 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:03:09.397896 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:03:09.400219 (+ 2323us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.192.254:37079: Server oryx/127.1.192.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":87,"threads_started":1}
W20260809 02:03:09.400489 11898 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:03:09.393937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.195:52539 (local address 127.1.192.254:37079)
0809 02:03:09.394115 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0809 02:03:09.394120 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:03:09.394289 (+ 169us) server_negotiation.cc:407] Connection header received
0809 02:03:09.394479 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:03:09.394482 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:03:09.394530 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:03:09.394626 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:03:09.395123 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:09.395798 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:09.396746 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:09.397036 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:09.397156 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:09.400369 (+ 3213us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.192.195:52539: BlockingRecv error: recv got EOF from 127.1.192.195:52539 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 09 02:03:09 dist-test-slave-ss4n krb5kdc[10795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.192.193@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:03:09 dist-test-slave-ss4n krb5kdc[10795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.192.193@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:03:10.002369 11903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:09.996075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:37079 (local address 127.1.192.193:39573)
0809 02:03:09.996469 (+ 394us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:09.996487 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 02:03:09.996614 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:03:09.996946 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:03:09.996949 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:03:09.997215 (+ 266us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:03:09.997345 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:09.997355 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:09.998407 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:09.998410 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:09.999014 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:09.999023 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:09.999506 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:09.999509 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:09.999579 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:10.000096 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:03:10.000108 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:03:10.002211 (+ 2103us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.192.254:37079: Server oryx/127.1.192.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":88,"threads_started":1}
W20260809 02:03:10.002580 11904 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:03:09.996211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.193:39573 (local address 127.1.192.254:37079)
0809 02:03:09.996490 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0809 02:03:09.996496 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:03:09.996600 (+ 104us) server_negotiation.cc:407] Connection header received
0809 02:03:09.996712 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:03:09.996714 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:03:09.996786 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:03:09.996889 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:03:09.997619 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:09.998214 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:09.999135 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:09.999369 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:09.999458 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:10.002427 (+ 2969us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.192.193:39573: BlockingRecv error: recv got EOF from 127.1.192.193:39573 (error 108)
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":97,"threads_started":1}
Aug 09 02:03:10 dist-test-slave-ss4n krb5kdc[10795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.192.194@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:03:10 dist-test-slave-ss4n krb5kdc[10795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.192.194@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:03:10.202150 11906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:10.192091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:37079 (local address 127.1.192.194:44075)
0809 02:03:10.192379 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:10.192394 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 02:03:10.192554 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:03:10.192799 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:03:10.192801 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:03:10.193006 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:03:10.193112 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:10.193118 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:10.193938 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:10.193941 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:10.194607 (+ 666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:10.194618 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:10.195083 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:10.195086 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:10.195169 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:10.199532 (+ 4363us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:03:10.199547 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:03:10.201942 (+ 2395us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.192.254:37079: Server oryx/127.1.192.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":106,"threads_started":1}
W20260809 02:03:10.202538 11904 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:03:10.192202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.194:44075 (local address 127.1.192.254:37079)
0809 02:03:10.192344 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0809 02:03:10.192347 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:03:10.192514 (+ 167us) server_negotiation.cc:407] Connection header received
0809 02:03:10.192640 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:03:10.192643 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:03:10.192686 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:03:10.192754 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:03:10.193225 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:10.193798 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:10.194729 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:10.194968 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:10.195047 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:10.202427 (+ 7380us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.192.194:44075: BlockingRecv error: recv got EOF from 127.1.192.194:44075 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 09 02:03:10 dist-test-slave-ss4n krb5kdc[10795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.192.195@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:03:10 dist-test-slave-ss4n krb5kdc[10795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.192.195@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:03:10.407226 11907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:10.401310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:37079 (local address 127.1.192.195:47279)
0809 02:03:10.401628 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:10.401647 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 02:03:10.401760 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:03:10.402097 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:03:10.402099 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:03:10.402295 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:03:10.402390 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:10.402395 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:10.403219 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:10.403222 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:10.403768 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:10.403776 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:10.404227 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:10.404230 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:10.404288 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:10.404831 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:03:10.404844 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:03:10.407077 (+ 2233us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.192.254:37079: Server oryx/127.1.192.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":123,"threads_started":1}
W20260809 02:03:10.407389 11904 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:03:10.401508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.195:47279 (local address 127.1.192.254:37079)
0809 02:03:10.401649 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0809 02:03:10.401653 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:03:10.401731 (+ 78us) server_negotiation.cc:407] Connection header received
0809 02:03:10.401896 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:03:10.401899 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:03:10.401952 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:03:10.402038 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:03:10.402515 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:10.403092 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:10.403892 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:10.404122 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:10.404209 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:10.407257 (+ 3048us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.192.195:47279: BlockingRecv error: recv got EOF from 127.1.192.195:47279 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 09 02:03:11 dist-test-slave-ss4n krb5kdc[10795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.192.193@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:03:11 dist-test-slave-ss4n krb5kdc[10795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.192.193@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:03:11.008955 11908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:11.003196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:37079 (local address 127.1.192.193:39971)
0809 02:03:11.003488 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:11.003502 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 02:03:11.003603 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:03:11.003836 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:03:11.003838 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:03:11.004025 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:03:11.004125 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:11.004131 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:11.005048 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:11.005050 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:11.005636 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:11.005644 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:11.006124 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:11.006126 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:11.006185 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:11.006862 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:03:11.006874 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:03:11.008830 (+ 1956us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.192.254:37079: Server oryx/127.1.192.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":76,"threads_started":1}
W20260809 02:03:11.009219 11909 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:03:11.003280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.193:39971 (local address 127.1.192.254:37079)
0809 02:03:11.003591 (+ 311us) server_negotiation.cc:206] Beginning negotiation
0809 02:03:11.003595 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:03:11.003609 (+ 14us) server_negotiation.cc:407] Connection header received
0809 02:03:11.003673 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:03:11.003675 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:03:11.003719 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:03:11.003795 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:03:11.004294 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:11.004936 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:11.005749 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:11.006011 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:11.006109 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:11.009081 (+ 2972us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.192.193:39971: BlockingRecv error: recv got EOF from 127.1.192.193:39971 (error 108)
Metrics: {"server-negotiator.queue_time_us":197,"thread_start_us":88,"threads_started":1}
Aug 09 02:03:11 dist-test-slave-ss4n krb5kdc[10795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.192.194@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:03:11 dist-test-slave-ss4n krb5kdc[10795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.192.194@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:03:11.209062 11910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:11.203011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:37079 (local address 127.1.192.194:37045)
0809 02:03:11.203426 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:11.203439 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:03:11.203573 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:03:11.203896 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:03:11.203899 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:03:11.204125 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:03:11.204220 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:11.204226 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:11.205055 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:11.205060 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:11.205655 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:11.205663 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:11.206139 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:11.206142 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:11.206231 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:11.206777 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:03:11.206788 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:03:11.208935 (+ 2147us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.192.254:37079: Server oryx/127.1.192.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":120,"threads_started":1}
W20260809 02:03:11.209357 11909 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:03:11.203186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.194:37045 (local address 127.1.192.254:37079)
0809 02:03:11.203335 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0809 02:03:11.203339 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:03:11.203528 (+ 189us) server_negotiation.cc:407] Connection header received
0809 02:03:11.203696 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:03:11.203701 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:03:11.203762 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:03:11.203867 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:03:11.204352 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:11.204929 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:11.205775 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:11.206021 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:11.206134 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:11.209091 (+ 2957us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.192.194:37045: BlockingRecv error: recv got EOF from 127.1.192.194:37045 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 09 02:03:11 dist-test-slave-ss4n krb5kdc[10795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.192.195@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:03:11 dist-test-slave-ss4n krb5kdc[10795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.192.195@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:03:11.414681 11911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:11.408202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:37079 (local address 127.1.192.195:45047)
0809 02:03:11.408639 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:11.408654 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 02:03:11.408815 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:03:11.409136 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:03:11.409140 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:03:11.409381 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:03:11.409522 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:11.409529 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:11.410374 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:11.410380 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:11.410999 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:11.411008 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:11.411578 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:11.411583 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:11.411754 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:11.412341 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:03:11.412355 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:03:11.414506 (+ 2151us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.192.254:37079: Server oryx/127.1.192.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":114,"threads_started":1}
W20260809 02:03:11.414785 11909 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:03:11.408258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.195:45047 (local address 127.1.192.254:37079)
0809 02:03:11.408425 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0809 02:03:11.408431 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:03:11.408744 (+ 313us) server_negotiation.cc:407] Connection header received
0809 02:03:11.408937 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:03:11.408942 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:03:11.408998 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:03:11.409105 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:03:11.409660 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:11.410243 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:11.411153 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:11.411396 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:11.411519 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:11.414660 (+ 3141us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.192.195:45047: BlockingRecv error: recv got EOF from 127.1.192.195:45047 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 09 02:03:12 dist-test-slave-ss4n krb5kdc[10795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.192.193@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:03:12 dist-test-slave-ss4n krb5kdc[10795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.192.193@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:03:12.015618 11912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:12.009809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:37079 (local address 127.1.192.193:36713)
0809 02:03:12.010093 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:12.010106 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:03:12.010195 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:03:12.010607 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:03:12.010609 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:03:12.010795 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:03:12.010893 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:12.010899 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:12.011781 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:12.011783 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:12.012336 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:12.012343 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:12.012831 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:12.012833 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:12.012896 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:12.013543 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:03:12.013560 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:03:12.015451 (+ 1891us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.192.254:37079: Server oryx/127.1.192.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":103,"threads_started":1}
W20260809 02:03:12.015854 11913 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:03:12.009917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.193:36713 (local address 127.1.192.254:37079)
0809 02:03:12.010292 (+ 375us) server_negotiation.cc:206] Beginning negotiation
0809 02:03:12.010296 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:03:12.010311 (+ 15us) server_negotiation.cc:407] Connection header received
0809 02:03:12.010354 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:03:12.010356 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:03:12.010410 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:03:12.010569 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:03:12.011009 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:12.011660 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:12.012457 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:12.012723 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:12.012813 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:12.015635 (+ 2822us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.192.193:36713: BlockingRecv error: recv got EOF from 127.1.192.193:36713 (error 108)
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":167,"threads_started":1}
Aug 09 02:03:12 dist-test-slave-ss4n krb5kdc[10795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.192.194@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:03:12 dist-test-slave-ss4n krb5kdc[10795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.192.194@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:03:12.215559 11914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:12.210004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:37079 (local address 127.1.192.194:45203)
0809 02:03:12.210356 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:12.210369 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:03:12.210463 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:03:12.210691 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:03:12.210693 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:03:12.210875 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:03:12.210974 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:12.210980 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:12.211728 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:12.211730 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:12.212285 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:12.212292 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:12.212732 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:12.212734 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:12.212796 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:12.213393 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:03:12.213405 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:03:12.215383 (+ 1978us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.192.254:37079: Server oryx/127.1.192.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":106,"threads_started":1}
W20260809 02:03:12.215638 11913 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:03:12.210074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.194:45203 (local address 127.1.192.254:37079)
0809 02:03:12.210202 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0809 02:03:12.210206 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:03:12.210422 (+ 216us) server_negotiation.cc:407] Connection header received
0809 02:03:12.210531 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:03:12.210539 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:03:12.210585 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:03:12.210652 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:03:12.211075 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:12.211625 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:12.212381 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:12.212629 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:12.212708 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:12.215510 (+ 2802us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.192.194:45203: BlockingRecv error: recv got EOF from 127.1.192.194:45203 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Aug 09 02:03:12 dist-test-slave-ss4n krb5kdc[10795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.192.195@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:03:12 dist-test-slave-ss4n krb5kdc[10795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.192.195@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:03:12.422005 11915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:12.415770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:37079 (local address 127.1.192.195:38527)
0809 02:03:12.416208 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:12.416228 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 02:03:12.416372 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:03:12.416766 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:03:12.416773 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:03:12.417002 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:03:12.417111 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:12.417117 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:12.418041 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:12.418044 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:12.418661 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:12.418671 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:12.419137 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:12.419140 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:12.419209 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:12.419735 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:03:12.419747 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:03:12.421862 (+ 2115us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.192.254:37079: Server oryx/127.1.192.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":160,"threads_started":1}
W20260809 02:03:12.422288 11913 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:03:12.415852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.195:38527 (local address 127.1.192.254:37079)
0809 02:03:12.416026 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0809 02:03:12.416031 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:03:12.416350 (+ 319us) server_negotiation.cc:407] Connection header received
0809 02:03:12.416503 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:03:12.416506 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:03:12.416572 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:03:12.416687 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:03:12.417241 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:12.417913 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:12.418788 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:12.419003 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:12.419092 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:12.422155 (+ 3063us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.192.195:38527: BlockingRecv error: recv got EOF from 127.1.192.195:38527 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 09 02:03:13 dist-test-slave-ss4n krb5kdc[10795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.192.193@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:03:13 dist-test-slave-ss4n krb5kdc[10795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.192.193@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:03:13.023444 11916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:13.016565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:37079 (local address 127.1.192.193:57421)
0809 02:03:13.016887 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:13.016904 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 02:03:13.017076 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:03:13.017370 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:03:13.017373 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:03:13.017682 (+ 309us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:03:13.017783 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:13.017789 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:13.018579 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:13.018585 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:13.019155 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:13.019163 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:13.019652 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:13.019656 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:13.019743 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:13.020498 (+ 755us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:03:13.020517 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:03:13.023198 (+ 2681us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.192.254:37079: Server oryx/127.1.192.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":108,"threads_started":1}
W20260809 02:03:13.023640 11917 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:03:13.016631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.193:57421 (local address 127.1.192.254:37079)
0809 02:03:13.016888 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0809 02:03:13.016894 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:03:13.017001 (+ 107us) server_negotiation.cc:407] Connection header received
0809 02:03:13.017193 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:03:13.017197 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:03:13.017249 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:03:13.017350 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:03:13.017911 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:13.018461 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:13.019282 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:13.019528 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:13.019647 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:13.023464 (+ 3817us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.192.193:57421: BlockingRecv error: recv got EOF from 127.1.192.193:57421 (error 108)
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":86,"threads_started":1}
Aug 09 02:03:13 dist-test-slave-ss4n krb5kdc[10795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.192.194@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:03:13 dist-test-slave-ss4n krb5kdc[10795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.192.194@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:03:13.222524 11918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:13.216453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:37079 (local address 127.1.192.194:60065)
0809 02:03:13.216823 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:13.216837 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 02:03:13.216937 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:03:13.217251 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:03:13.217254 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:03:13.217467 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:03:13.217610 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:13.217617 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:13.218426 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:13.218429 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:13.219055 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:13.219067 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:13.219615 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:13.219617 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:13.219682 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:13.220313 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:03:13.220325 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:03:13.222406 (+ 2081us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.192.254:37079: Server oryx/127.1.192.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":180,"threads_started":1}
W20260809 02:03:13.222698 11917 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:03:13.216656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.194:60065 (local address 127.1.192.254:37079)
0809 02:03:13.216808 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0809 02:03:13.216811 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:03:13.216911 (+ 100us) server_negotiation.cc:407] Connection header received
0809 02:03:13.217023 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:03:13.217025 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:03:13.217085 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:03:13.217157 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:03:13.217753 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:13.218302 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:13.219182 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:13.219455 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:13.219557 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:13.222564 (+ 3007us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.192.194:60065: BlockingRecv error: recv got EOF from 127.1.192.194:60065 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 09 02:03:13 dist-test-slave-ss4n krb5kdc[10795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.192.195@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:03:13 dist-test-slave-ss4n krb5kdc[10795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.192.195@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:03:13.429129 11919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:13.423011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:37079 (local address 127.1.192.195:53883)
0809 02:03:13.423289 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:13.423302 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:03:13.423401 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:03:13.423723 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:03:13.423725 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:03:13.423915 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:03:13.424032 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:13.424037 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:13.424869 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:13.424872 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:13.425465 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:13.425473 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:13.425993 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:13.425996 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:13.426086 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:13.426658 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:03:13.426670 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:03:13.429002 (+ 2332us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.192.254:37079: Server oryx/127.1.192.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":108,"threads_started":1}
W20260809 02:03:13.429276 11917 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:03:13.423128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.195:53883 (local address 127.1.192.254:37079)
0809 02:03:13.423278 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0809 02:03:13.423282 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:03:13.423377 (+ 95us) server_negotiation.cc:407] Connection header received
0809 02:03:13.423530 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:03:13.423532 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:03:13.423576 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:03:13.423649 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:03:13.424200 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:13.424757 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:13.425635 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:13.425867 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:13.425964 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:13.429148 (+ 3184us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.192.195:53883: BlockingRecv error: recv got EOF from 127.1.192.195:53883 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 09 02:03:14 dist-test-slave-ss4n krb5kdc[10795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.192.193@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:03:14 dist-test-slave-ss4n krb5kdc[10795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.192.193@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:03:14.030253 11920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:14.024388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:37079 (local address 127.1.192.193:33935)
0809 02:03:14.024664 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:14.024681 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 02:03:14.024790 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:03:14.025075 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:03:14.025077 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:03:14.025270 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:03:14.025368 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:14.025373 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:14.026222 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:14.026224 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:14.026867 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:14.026877 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:14.027469 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:14.027472 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:14.027535 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:14.028088 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:03:14.028100 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:03:14.030133 (+ 2033us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.192.254:37079: Server oryx/127.1.192.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":100,"threads_started":1}
W20260809 02:03:14.030589 11921 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:03:14.024454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.193:33935 (local address 127.1.192.254:37079)
0809 02:03:14.024719 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0809 02:03:14.024728 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 02:03:14.024742 (+ 14us) server_negotiation.cc:407] Connection header received
0809 02:03:14.024877 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:03:14.024879 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:03:14.024922 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:03:14.025012 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:03:14.025555 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:14.026109 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:14.027066 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:14.027354 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:14.027504 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:14.030317 (+ 2813us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.192.193:33935: BlockingRecv error: recv got EOF from 127.1.192.193:33935 (error 108)
Metrics: {"server-negotiator.queue_time_us":197,"thread_start_us":109,"threads_started":1}
Aug 09 02:03:14 dist-test-slave-ss4n krb5kdc[10795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.192.194@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:03:14 dist-test-slave-ss4n krb5kdc[10795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.192.194@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:03:14.229918 11922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:14.223644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:37079 (local address 127.1.192.194:54509)
0809 02:03:14.223986 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:14.224001 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 02:03:14.224140 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:03:14.224419 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:03:14.224422 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:03:14.224649 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:03:14.224753 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:14.224759 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:14.225844 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:14.225849 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:14.226415 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:14.226423 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:14.226898 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:14.226901 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:14.226987 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:14.227636 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:03:14.227648 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:03:14.229797 (+ 2149us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.192.254:37079: Server oryx/127.1.192.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":85,"threads_started":1}
W20260809 02:03:14.230247 11921 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:03:14.223680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.194:54509 (local address 127.1.192.254:37079)
0809 02:03:14.223855 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0809 02:03:14.223861 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:03:14.224093 (+ 232us) server_negotiation.cc:407] Connection header received
0809 02:03:14.224247 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:03:14.224251 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:03:14.224305 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:03:14.224394 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:03:14.224876 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:14.225725 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:14.226540 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:14.226778 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:14.226891 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:14.230108 (+ 3217us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.192.194:54509: BlockingRecv error: recv got EOF from 127.1.192.194:54509 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 09 02:03:14 dist-test-slave-ss4n krb5kdc[10795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.192.195@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:03:14 dist-test-slave-ss4n krb5kdc[10795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.192.195@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:03:14.436673 11923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:14.430234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:37079 (local address 127.1.192.195:54897)
0809 02:03:14.430632 (+ 398us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:14.430672 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 02:03:14.430801 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:03:14.431185 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:03:14.431188 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:03:14.431438 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:03:14.431671 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:14.431680 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:14.432645 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:14.432648 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:14.433297 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:14.433307 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:14.433799 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:14.433802 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:14.433876 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:14.434420 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:03:14.434434 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:03:14.436541 (+ 2107us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.192.254:37079: Server oryx/127.1.192.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":113,"threads_started":1}
W20260809 02:03:14.436863 11921 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:03:14.430365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.195:54897 (local address 127.1.192.254:37079)
0809 02:03:14.430507 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0809 02:03:14.430515 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 02:03:14.430777 (+ 262us) server_negotiation.cc:407] Connection header received
0809 02:03:14.430948 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:03:14.430951 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:03:14.431030 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:03:14.431128 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:03:14.431839 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:14.432443 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:14.433410 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:14.433677 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:14.433766 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:14.436698 (+ 2932us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.192.195:54897: BlockingRecv error: recv got EOF from 127.1.192.195:54897 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 09 02:03:15 dist-test-slave-ss4n krb5kdc[10795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.192.193@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:03:15 dist-test-slave-ss4n krb5kdc[10795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.192.193@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:03:15.037698 11924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:15.031220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:37079 (local address 127.1.192.193:37347)
0809 02:03:15.031658 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:15.031673 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 02:03:15.031780 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:03:15.032116 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:03:15.032119 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:03:15.032322 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:03:15.032439 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:15.032445 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:15.033832 (+ 1387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:15.033839 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:15.034430 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:15.034441 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:15.034938 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:15.034941 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:15.035009 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:15.035625 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:03:15.035638 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:03:15.037573 (+ 1935us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.192.254:37079: Server oryx/127.1.192.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":127,"threads_started":1}
W20260809 02:03:15.037935 11925 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:03:15.031315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.193:37347 (local address 127.1.192.254:37079)
0809 02:03:15.031732 (+ 417us) server_negotiation.cc:206] Beginning negotiation
0809 02:03:15.031736 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:03:15.031767 (+ 31us) server_negotiation.cc:407] Connection header received
0809 02:03:15.031914 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:03:15.031917 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:03:15.031979 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:03:15.032078 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:03:15.032750 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:15.033671 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:15.034575 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:15.034822 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:15.034914 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:15.037765 (+ 2851us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.192.193:37347: BlockingRecv error: recv got EOF from 127.1.192.193:37347 (error 108)
Metrics: {"server-negotiator.queue_time_us":327,"thread_start_us":220,"threads_started":1}
Aug 09 02:03:15 dist-test-slave-ss4n krb5kdc[10795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.192.194@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:03:15 dist-test-slave-ss4n krb5kdc[10795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.192.194@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:03:15.240381 11926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:15.230783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:37079 (local address 127.1.192.194:51725)
0809 02:03:15.231207 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:15.231221 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 02:03:15.231378 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:03:15.231722 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:03:15.231726 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:03:15.231965 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:03:15.232101 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:15.232114 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:15.232980 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:15.232984 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:15.233700 (+ 716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:15.233711 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:15.234199 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:15.234203 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:15.234295 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:15.237898 (+ 3603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:03:15.237915 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:03:15.240263 (+ 2348us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.192.254:37079: Server oryx/127.1.192.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":113,"threads_started":1}
W20260809 02:03:15.240600 11925 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:03:15.230908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.194:51725 (local address 127.1.192.254:37079)
0809 02:03:15.231129 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0809 02:03:15.231136 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 02:03:15.231312 (+ 176us) server_negotiation.cc:407] Connection header received
0809 02:03:15.231538 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:03:15.231541 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:03:15.231602 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:03:15.231702 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:03:15.232278 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:15.232833 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:15.233828 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:15.234071 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:15.234196 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:15.240422 (+ 6226us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.192.194:51725: BlockingRecv error: recv got EOF from 127.1.192.194:51725 (error 108)
Metrics: {"server-negotiator.queue_time_us":106}
Aug 09 02:03:15 dist-test-slave-ss4n krb5kdc[10795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.192.195@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:03:15 dist-test-slave-ss4n krb5kdc[10795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.192.195@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:03:15.444218 11927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:15.437607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:37079 (local address 127.1.192.195:37893)
0809 02:03:15.438017 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:15.438051 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 02:03:15.438163 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:03:15.438403 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:03:15.438405 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:03:15.438628 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:03:15.438731 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:15.438737 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:15.439623 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:15.439626 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:15.440237 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:15.440246 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:15.440812 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:15.440815 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:15.440884 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:15.441563 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:03:15.441578 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:03:15.444033 (+ 2455us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.192.254:37079: Server oryx/127.1.192.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":168,"threads_started":1}
W20260809 02:03:15.444289 11925 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:03:15.437661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.195:37893 (local address 127.1.192.254:37079)
0809 02:03:15.437819 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0809 02:03:15.437824 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:03:15.438131 (+ 307us) server_negotiation.cc:407] Connection header received
0809 02:03:15.438252 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:03:15.438255 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:03:15.438308 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:03:15.438392 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:03:15.438856 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:15.439454 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:15.440378 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:15.440669 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:15.440805 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:15.444173 (+ 3368us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.192.195:37893: BlockingRecv error: recv got EOF from 127.1.192.195:37893 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 09 02:03:16 dist-test-slave-ss4n krb5kdc[10795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.192.193@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:03:16 dist-test-slave-ss4n krb5kdc[10795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.192.193@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:03:16.045095 11928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:16.038642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:37079 (local address 127.1.192.193:44899)
0809 02:03:16.039086 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:16.039105 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 02:03:16.039221 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:03:16.039539 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:03:16.039585 (+ 46us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:03:16.039788 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:03:16.039926 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:16.039937 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:16.040822 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:16.040825 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:16.041444 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:16.041453 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:16.042035 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:16.042039 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:16.042138 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:16.042723 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:03:16.042736 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:03:16.044956 (+ 2220us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.192.254:37079: Server oryx/127.1.192.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":93,"threads_started":1}
W20260809 02:03:16.045328 11929 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:03:16.038805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.193:44899 (local address 127.1.192.254:37079)
0809 02:03:16.039091 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0809 02:03:16.039097 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:03:16.039191 (+ 94us) server_negotiation.cc:407] Connection header received
0809 02:03:16.039319 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:03:16.039323 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:03:16.039382 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:03:16.039464 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:03:16.040079 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:16.040697 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:16.041634 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:16.041900 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:16.042026 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:16.045139 (+ 3113us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.192.193:44899: BlockingRecv error: recv got EOF from 127.1.192.193:44899 (error 108)
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":78,"threads_started":1}
Aug 09 02:03:16 dist-test-slave-ss4n krb5kdc[10795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.192.194@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:03:16 dist-test-slave-ss4n krb5kdc[10795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.192.194@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:03:16.248256 11930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:16.241334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:37079 (local address 127.1.192.194:51677)
0809 02:03:16.241744 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:16.241764 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 02:03:16.241903 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:03:16.242280 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:03:16.242283 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:03:16.242521 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:03:16.242626 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:16.242632 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:16.243821 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:16.243824 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:16.244609 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:16.244621 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:16.245199 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:16.245202 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:16.245278 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:16.245936 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:03:16.245950 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:03:16.248127 (+ 2177us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.192.254:37079: Server oryx/127.1.192.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":147,"threads_started":1}
W20260809 02:03:16.248404 11929 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:03:16.241536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.194:51677 (local address 127.1.192.254:37079)
0809 02:03:16.241759 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0809 02:03:16.241766 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 02:03:16.241839 (+ 73us) server_negotiation.cc:407] Connection header received
0809 02:03:16.242041 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:03:16.242046 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:03:16.242116 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:03:16.242240 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:03:16.242767 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:16.243686 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:16.244709 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:16.245051 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:16.245149 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:16.248284 (+ 3135us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.192.194:51677: BlockingRecv error: recv got EOF from 127.1.192.194:51677 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 09 02:03:16 dist-test-slave-ss4n krb5kdc[10795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.192.195@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:03:16 dist-test-slave-ss4n krb5kdc[10795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.192.195@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:03:16.451972 11931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:16.445379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:37079 (local address 127.1.192.195:48383)
0809 02:03:16.445776 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:16.445802 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0809 02:03:16.445938 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:03:16.446277 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:03:16.446280 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:03:16.446486 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:03:16.446648 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:16.446655 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:16.447463 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:16.447466 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:16.448170 (+ 704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:16.448181 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:16.448737 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:16.448740 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:16.448812 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:16.449370 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:03:16.449383 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:03:16.451800 (+ 2417us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.192.254:37079: Server oryx/127.1.192.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":97,"threads_started":1}
W20260809 02:03:16.452103 11929 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:03:16.445647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.195:48383 (local address 127.1.192.254:37079)
0809 02:03:16.445804 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0809 02:03:16.445817 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 02:03:16.445936 (+ 119us) server_negotiation.cc:407] Connection header received
0809 02:03:16.446040 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:03:16.446042 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:03:16.446103 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:03:16.446186 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:03:16.446786 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:16.447342 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:16.448308 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:16.448577 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:16.448666 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:16.451958 (+ 3292us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.192.195:48383: BlockingRecv error: recv got EOF from 127.1.192.195:48383 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 09 02:03:17 dist-test-slave-ss4n krb5kdc[10795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.192.193@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:03:17 dist-test-slave-ss4n krb5kdc[10795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.192.193@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:03:17.052242 11932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:17.046145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:37079 (local address 127.1.192.193:41875)
0809 02:03:17.046508 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:17.046525 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 02:03:17.046638 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:03:17.046926 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:03:17.046929 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:03:17.047184 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:03:17.047304 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:17.047310 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:17.048105 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:17.048110 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:17.048679 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:17.048687 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:17.049176 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:17.049180 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:17.049277 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:17.049814 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:03:17.049826 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:03:17.052081 (+ 2255us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.192.254:37079: Server oryx/127.1.192.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":106,"threads_started":1}
W20260809 02:03:17.052412 11933 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:03:17.046196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.193:41875 (local address 127.1.192.254:37079)
0809 02:03:17.046617 (+ 421us) server_negotiation.cc:206] Beginning negotiation
0809 02:03:17.046622 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:03:17.046635 (+ 13us) server_negotiation.cc:407] Connection header received
0809 02:03:17.046730 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:03:17.046735 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:03:17.046800 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:03:17.046903 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:03:17.047430 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:17.047989 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:17.048816 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:17.049057 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:17.049171 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:17.052259 (+ 3088us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.192.193:41875: BlockingRecv error: recv got EOF from 127.1.192.193:41875 (error 108)
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":88,"threads_started":1}
Aug 09 02:03:17 dist-test-slave-ss4n krb5kdc[10795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.192.194@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:03:17 dist-test-slave-ss4n krb5kdc[10795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.192.194@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:03:17.255090 11934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:17.249200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:37079 (local address 127.1.192.194:38379)
0809 02:03:17.249537 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:17.249555 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 02:03:17.249664 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:03:17.249927 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:03:17.249929 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:03:17.250142 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:03:17.250241 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:17.250246 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:17.251151 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:17.251155 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:17.251783 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:17.251793 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:17.252266 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:17.252270 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:17.252361 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:17.252894 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:03:17.252906 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:03:17.254937 (+ 2031us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.192.254:37079: Server oryx/127.1.192.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":113,"threads_started":1}
W20260809 02:03:17.255227 11933 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:03:17.249372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.194:38379 (local address 127.1.192.254:37079)
0809 02:03:17.249589 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0809 02:03:17.249592 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:03:17.249609 (+ 17us) server_negotiation.cc:407] Connection header received
0809 02:03:17.249764 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:03:17.249766 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:03:17.249812 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:03:17.249889 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:03:17.250400 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:17.251024 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:17.251915 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:17.252142 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:17.252249 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:17.255084 (+ 2835us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.192.194:38379: BlockingRecv error: recv got EOF from 127.1.192.194:38379 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 09 02:03:17 dist-test-slave-ss4n krb5kdc[10795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.192.195@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:03:17 dist-test-slave-ss4n krb5kdc[10795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.192.195@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:03:17.459240 11935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:17.453074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:37079 (local address 127.1.192.195:41519)
0809 02:03:17.453365 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:17.453379 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 02:03:17.453509 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:03:17.453827 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:03:17.453829 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:03:17.454020 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:03:17.454124 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:17.454130 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:17.455146 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:17.455151 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:17.455807 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:17.455816 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:17.456362 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:17.456366 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:17.456454 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:17.457050 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:03:17.457063 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:03:17.459043 (+ 1980us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.192.254:37079: Server oryx/127.1.192.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":79,"threads_started":1}
W20260809 02:03:17.459278 11933 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:03:17.453165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.195:41519 (local address 127.1.192.254:37079)
0809 02:03:17.453362 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0809 02:03:17.453367 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:03:17.453457 (+ 90us) server_negotiation.cc:407] Connection header received
0809 02:03:17.453651 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:03:17.453654 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:03:17.453704 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:03:17.453777 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:03:17.454247 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:17.455048 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:17.455930 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:17.456180 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:17.456340 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:17.459176 (+ 2836us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.192.195:41519: BlockingRecv error: recv got EOF from 127.1.192.195:41519 (error 108)
Metrics: {"server-negotiator.queue_time_us":96}
Aug 09 02:03:18 dist-test-slave-ss4n krb5kdc[10795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.192.193@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:03:18 dist-test-slave-ss4n krb5kdc[10795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.192.193@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:03:18.059340 11936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:18.053265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:37079 (local address 127.1.192.193:43027)
0809 02:03:18.053576 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:18.053595 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 02:03:18.053699 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:03:18.053941 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:03:18.053943 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:03:18.054149 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:03:18.054251 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:18.054256 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:18.055106 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:18.055113 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:18.055974 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:18.055987 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:18.056589 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:18.056592 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:18.056658 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:18.057254 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:03:18.057276 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:03:18.059216 (+ 1940us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.192.254:37079: Server oryx/127.1.192.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":99,"threads_started":1}
W20260809 02:03:18.059514 11937 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:03:18.053370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.193:43027 (local address 127.1.192.254:37079)
0809 02:03:18.053650 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0809 02:03:18.053653 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:03:18.053670 (+ 17us) server_negotiation.cc:407] Connection header received
0809 02:03:18.053773 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:03:18.053775 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:03:18.053817 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:03:18.053905 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:03:18.054363 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:18.054975 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:18.056125 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:18.056468 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:18.056557 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:18.059372 (+ 2815us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.192.193:43027: BlockingRecv error: recv got EOF from 127.1.192.193:43027 (error 108)
Metrics: {"server-negotiator.queue_time_us":200,"thread_start_us":136,"threads_started":1}
Aug 09 02:03:18 dist-test-slave-ss4n krb5kdc[10795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.192.194@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:03:18 dist-test-slave-ss4n krb5kdc[10795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.192.194@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:03:18.262445 11938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:18.256103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:37079 (local address 127.1.192.194:54703)
0809 02:03:18.256447 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:18.256469 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 02:03:18.256613 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:03:18.256933 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:03:18.256958 (+ 25us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:03:18.257160 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:03:18.257266 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:18.257272 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:18.258087 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:18.258090 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:18.258875 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:18.258887 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:18.259516 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:18.259520 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:18.259615 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:18.260272 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:03:18.260289 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:03:18.262315 (+ 2026us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.192.254:37079: Server oryx/127.1.192.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":137,"threads_started":1}
W20260809 02:03:18.262895 11937 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:03:18.256197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.194:54703 (local address 127.1.192.254:37079)
0809 02:03:18.256372 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0809 02:03:18.256376 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:03:18.256596 (+ 220us) server_negotiation.cc:407] Connection header received
0809 02:03:18.256714 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:03:18.256716 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:03:18.256768 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:03:18.256850 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:03:18.257384 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:18.257971 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:18.259034 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:18.259379 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:18.259492 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:18.262777 (+ 3285us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.192.194:54703: BlockingRecv error: recv got EOF from 127.1.192.194:54703 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 09 02:03:18 dist-test-slave-ss4n krb5kdc[10795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.192.195@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:03:18 dist-test-slave-ss4n krb5kdc[10795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.192.195@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:03:18.466884 11939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:18.460346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:37079 (local address 127.1.192.195:40171)
0809 02:03:18.460707 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:18.460721 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 02:03:18.460818 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:03:18.461113 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:03:18.461116 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:03:18.461307 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:03:18.461424 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:18.461430 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:18.462335 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:18.462338 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:18.462952 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:18.462962 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:18.463588 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:18.463592 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:18.463658 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:18.464199 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:03:18.464210 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:03:18.466686 (+ 2476us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.192.254:37079: Server oryx/127.1.192.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":133,"threads_started":1}
W20260809 02:03:18.466964 11937 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:03:18.460567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.195:40171 (local address 127.1.192.254:37079)
0809 02:03:18.460710 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0809 02:03:18.460714 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:03:18.460778 (+ 64us) server_negotiation.cc:407] Connection header received
0809 02:03:18.460889 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:03:18.460892 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:03:18.460943 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:03:18.461031 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:03:18.461615 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:18.462217 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:18.463097 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:18.463392 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:18.463539 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:18.466869 (+ 3330us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.192.195:40171: BlockingRecv error: recv got EOF from 127.1.192.195:40171 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 09 02:03:19 dist-test-slave-ss4n krb5kdc[10795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.192.193@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:03:19 dist-test-slave-ss4n krb5kdc[10795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.192.193@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:03:19.072031 11940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:19.060420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:37079 (local address 127.1.192.193:48341)
0809 02:03:19.060788 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:19.060812 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 02:03:19.060909 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:03:19.061156 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:03:19.061159 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:03:19.061353 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:03:19.061458 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:19.061463 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:19.062320 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:19.062322 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:19.062947 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:19.062957 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:19.063415 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:19.063418 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:19.063486 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:19.069466 (+ 5980us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:03:19.069514 (+ 48us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:03:19.071908 (+ 2394us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.192.254:37079: Server oryx/127.1.192.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":170,"threads_started":1}
W20260809 02:03:19.072386 11941 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:03:19.060523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.193:48341 (local address 127.1.192.254:37079)
0809 02:03:19.060929 (+ 406us) server_negotiation.cc:206] Beginning negotiation
0809 02:03:19.060932 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:03:19.060945 (+ 13us) server_negotiation.cc:407] Connection header received
0809 02:03:19.060982 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:03:19.060984 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:03:19.061024 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:03:19.061116 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:03:19.061671 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:19.062213 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:19.063084 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:19.063308 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:19.063390 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:19.072078 (+ 8688us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.192.193:48341: BlockingRecv error: recv got EOF from 127.1.192.193:48341 (error 108)
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":204,"threads_started":1}
Aug 09 02:03:19 dist-test-slave-ss4n krb5kdc[10795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.192.194@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:03:19 dist-test-slave-ss4n krb5kdc[10795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.192.194@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:03:19.269682 11942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:19.263358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:37079 (local address 127.1.192.194:59679)
0809 02:03:19.263703 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:19.263726 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 02:03:19.263828 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:03:19.264170 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:03:19.264173 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:03:19.264368 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:03:19.264526 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:19.264533 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:19.265323 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:19.265326 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:19.266004 (+ 678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:19.266014 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:19.266560 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:19.266563 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:19.266635 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:19.267217 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:03:19.267229 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:03:19.269455 (+ 2226us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.192.254:37079: Server oryx/127.1.192.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":163,"threads_started":1}
W20260809 02:03:19.269821 11941 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:03:19.263492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.194:59679 (local address 127.1.192.254:37079)
0809 02:03:19.263640 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0809 02:03:19.263645 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:03:19.263806 (+ 161us) server_negotiation.cc:407] Connection header received
0809 02:03:19.263927 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:03:19.263930 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:03:19.264018 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:03:19.264122 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:03:19.264655 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:19.265206 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:19.266140 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:19.266381 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:19.266494 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:19.269704 (+ 3210us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.192.194:59679: BlockingRecv error: recv got EOF from 127.1.192.194:59679 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Aug 09 02:03:19 dist-test-slave-ss4n krb5kdc[10795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.192.195@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:03:19 dist-test-slave-ss4n krb5kdc[10795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.192.195@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:03:19.474200 11943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:19.467815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:37079 (local address 127.1.192.195:33861)
0809 02:03:19.468243 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:19.468262 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 02:03:19.468426 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:03:19.468732 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:03:19.468735 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:03:19.468954 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:03:19.469140 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:19.469149 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:19.470070 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:19.470076 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:19.470793 (+ 717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:19.470802 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:19.471283 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:19.471287 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:19.471379 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:19.471982 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:03:19.471995 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:03:19.474025 (+ 2030us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.192.254:37079: Server oryx/127.1.192.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":121,"threads_started":1}
W20260809 02:03:19.474305 11941 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:03:19.467985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.195:33861 (local address 127.1.192.254:37079)
0809 02:03:19.468196 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0809 02:03:19.468202 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:03:19.468361 (+ 159us) server_negotiation.cc:407] Connection header received
0809 02:03:19.468536 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:03:19.468539 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:03:19.468599 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:03:19.468703 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:03:19.469265 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:19.469949 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:19.470915 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:19.471159 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:19.471275 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:19.474178 (+ 2903us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.192.195:33861: BlockingRecv error: recv got EOF from 127.1.192.195:33861 (error 108)
Metrics: {"server-negotiator.queue_time_us":104}
Aug 09 02:03:20 dist-test-slave-ss4n krb5kdc[10795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.192.193@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:03:20 dist-test-slave-ss4n krb5kdc[10795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.192.193@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:03:20.079044 11944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:20.072877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:37079 (local address 127.1.192.193:34457)
0809 02:03:20.073227 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:20.073242 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 02:03:20.073365 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:03:20.073700 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:03:20.073702 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:03:20.073904 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:03:20.074011 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:20.074016 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:20.074889 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:20.074893 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:20.075657 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:20.075673 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:20.076229 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:20.076232 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:20.076319 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:20.076982 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:03:20.076994 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:03:20.078861 (+ 1867us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.192.254:37079: Server oryx/127.1.192.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":110,"threads_started":1}
W20260809 02:03:20.079121 11945 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:03:20.073004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.193:34457 (local address 127.1.192.254:37079)
0809 02:03:20.073258 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0809 02:03:20.073262 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:03:20.073320 (+ 58us) server_negotiation.cc:407] Connection header received
0809 02:03:20.073464 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:03:20.073467 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:03:20.073569 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:03:20.073652 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:03:20.074137 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:20.074728 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:20.075803 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:20.076098 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:20.076220 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:20.078990 (+ 2770us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.192.193:34457: BlockingRecv error: recv got EOF from 127.1.192.193:34457 (error 108)
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":66,"threads_started":1}
Aug 09 02:03:20 dist-test-slave-ss4n krb5kdc[10795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.192.194@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:03:20 dist-test-slave-ss4n krb5kdc[10795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.192.194@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:03:20.280241 11946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:20.270546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:37079 (local address 127.1.192.194:32779)
0809 02:03:20.270843 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:20.270856 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:03:20.270989 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:03:20.271327 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:03:20.271330 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:03:20.271555 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:03:20.271658 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:20.271664 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:20.272441 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:20.272446 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:20.273016 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:20.273024 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:20.273571 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:20.273575 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:20.273657 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:20.277842 (+ 4185us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:03:20.277856 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:03:20.280045 (+ 2189us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.192.254:37079: Server oryx/127.1.192.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":99,"threads_started":1}
W20260809 02:03:20.280380 11945 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:03:20.270592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.194:32779 (local address 127.1.192.254:37079)
0809 02:03:20.270723 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0809 02:03:20.270728 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:03:20.270938 (+ 210us) server_negotiation.cc:407] Connection header received
0809 02:03:20.271105 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:03:20.271109 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:03:20.271203 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:03:20.271298 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:03:20.271785 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:20.272327 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:20.273151 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:20.273421 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:20.273576 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:20.280246 (+ 6670us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.192.194:32779: BlockingRecv error: recv got EOF from 127.1.192.194:32779 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Aug 09 02:03:20 dist-test-slave-ss4n krb5kdc[10795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.192.195@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:03:20 dist-test-slave-ss4n krb5kdc[10795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.192.195@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:03:20.481905 11947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:20.475208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:37079 (local address 127.1.192.195:52199)
0809 02:03:20.475670 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:20.475689 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 02:03:20.475839 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:03:20.476174 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:03:20.476178 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:03:20.476462 (+ 284us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:03:20.476599 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:20.476608 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:20.477792 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:20.477795 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:20.478442 (+ 647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:20.478451 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:20.479077 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:20.479080 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:20.479153 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:20.479829 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:03:20.479841 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:03:20.481784 (+ 1943us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.192.254:37079: Server oryx/127.1.192.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":139,"threads_started":1}
W20260809 02:03:20.482129 11945 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:03:20.475470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.195:52199 (local address 127.1.192.254:37079)
0809 02:03:20.475672 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0809 02:03:20.475676 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:03:20.475802 (+ 126us) server_negotiation.cc:407] Connection header received
0809 02:03:20.475942 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:03:20.475945 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:03:20.476011 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:03:20.476145 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:03:20.476766 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:20.477662 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:20.478645 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:20.478926 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:20.479021 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:20.481957 (+ 2936us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.192.195:52199: BlockingRecv error: recv got EOF from 127.1.192.195:52199 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Aug 09 02:03:21 dist-test-slave-ss4n krb5kdc[10795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.192.193@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:03:21 dist-test-slave-ss4n krb5kdc[10795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.192.193@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:03:21.086012 11948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:21.079877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:37079 (local address 127.1.192.193:57127)
0809 02:03:21.080242 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:21.080257 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 02:03:21.080361 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:03:21.080641 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:03:21.080643 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:03:21.080832 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:03:21.080932 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:21.080938 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:21.081737 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:21.081740 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:21.082326 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:21.082335 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:21.082854 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:21.082856 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:21.082922 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:21.083567 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:03:21.083580 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:03:21.085854 (+ 2274us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.192.254:37079: Server oryx/127.1.192.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":146,"threads_started":1}
W20260809 02:03:21.086189 11949 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:03:21.079994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.193:57127 (local address 127.1.192.254:37079)
0809 02:03:21.080262 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0809 02:03:21.080265 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:03:21.080311 (+ 46us) server_negotiation.cc:407] Connection header received
0809 02:03:21.080449 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:03:21.080451 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:03:21.080497 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:03:21.080582 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:03:21.081037 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:21.081635 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:21.082471 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:21.082731 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:21.082836 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:21.086042 (+ 3206us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.192.193:57127: BlockingRecv error: recv got EOF from 127.1.192.193:57127 (error 108)
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":65,"threads_started":1}
Aug 09 02:03:21 dist-test-slave-ss4n krb5kdc[10795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.192.194@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:03:21 dist-test-slave-ss4n krb5kdc[10795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.192.194@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:03:21.288275 11950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:21.281312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:37079 (local address 127.1.192.194:51779)
0809 02:03:21.281817 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:21.281836 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 02:03:21.281972 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:03:21.282383 (+ 411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:03:21.282386 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:03:21.282706 (+ 320us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:03:21.282850 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:21.282858 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:21.284047 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:21.284050 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:21.284762 (+ 712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:21.284774 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:21.285306 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:21.285309 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:21.285379 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:21.285999 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:03:21.286014 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:03:21.288141 (+ 2127us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.192.254:37079: Server oryx/127.1.192.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":166,"threads_started":1}
W20260809 02:03:21.288463 11949 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:03:21.281640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.194:51779 (local address 127.1.192.254:37079)
0809 02:03:21.281815 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0809 02:03:21.281819 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:03:21.281928 (+ 109us) server_negotiation.cc:407] Connection header received
0809 02:03:21.282117 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:03:21.282121 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:03:21.282206 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:03:21.282309 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:03:21.282990 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:21.283901 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:21.284915 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:21.285176 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:21.285280 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:21.288318 (+ 3038us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.192.194:51779: BlockingRecv error: recv got EOF from 127.1.192.194:51779 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 09 02:03:21 dist-test-slave-ss4n krb5kdc[10795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.192.195@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:03:21 dist-test-slave-ss4n krb5kdc[10795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.192.195@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:03:21.489359 11952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:21.482861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:37079 (local address 127.1.192.195:46385)
0809 02:03:21.483186 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:21.483201 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 02:03:21.483307 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:03:21.483661 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:03:21.483664 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:03:21.483871 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:03:21.483972 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:21.483978 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:21.484773 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:21.484776 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:21.485414 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:21.485424 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:21.486093 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:21.486096 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:21.486170 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:21.486813 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:03:21.486827 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:03:21.489144 (+ 2317us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.192.254:37079: Server oryx/127.1.192.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":134,"threads_started":1}
W20260809 02:03:21.489428 11949 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:03:21.482977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.195:46385 (local address 127.1.192.254:37079)
0809 02:03:21.483115 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0809 02:03:21.483119 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:03:21.483275 (+ 156us) server_negotiation.cc:407] Connection header received
0809 02:03:21.483434 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:03:21.483437 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:03:21.483496 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:03:21.483588 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:03:21.484094 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:21.484648 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:21.485693 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:21.485932 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:21.486036 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:21.489327 (+ 3291us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.192.195:46385: BlockingRecv error: recv got EOF from 127.1.192.195:46385 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 09 02:03:22 dist-test-slave-ss4n krb5kdc[10795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.192.193@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:03:22 dist-test-slave-ss4n krb5kdc[10795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.192.193@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:03:22.092792 11953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:22.086991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:37079 (local address 127.1.192.193:51743)
0809 02:03:22.087346 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:22.087360 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 02:03:22.087445 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:03:22.087670 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:03:22.087672 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:03:22.087888 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:03:22.087984 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:22.087989 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:22.088766 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:22.088769 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:22.089388 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:22.089399 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:22.089927 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:22.089930 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:22.090012 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:22.090503 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:03:22.090518 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:03:22.092577 (+ 2059us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.192.254:37079: Server oryx/127.1.192.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":113,"threads_started":1}
W20260809 02:03:22.092875 11954 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:03:22.087019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.193:51743 (local address 127.1.192.254:37079)
0809 02:03:22.087388 (+ 369us) server_negotiation.cc:206] Beginning negotiation
0809 02:03:22.087392 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:03:22.087405 (+ 13us) server_negotiation.cc:407] Connection header received
0809 02:03:22.087533 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:03:22.087535 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:03:22.087580 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:03:22.087661 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:03:22.088086 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:22.088631 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:22.089557 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:22.089793 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:22.089878 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:22.092730 (+ 2852us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.192.193:51743: BlockingRecv error: recv got EOF from 127.1.192.193:51743 (error 108)
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":80,"threads_started":1}
Aug 09 02:03:22 dist-test-slave-ss4n krb5kdc[10795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.192.194@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:03:22 dist-test-slave-ss4n krb5kdc[10795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.192.194@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:03:22.296006 11955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:22.289473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:37079 (local address 127.1.192.194:37963)
0809 02:03:22.289835 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:22.289850 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 02:03:22.290014 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:03:22.290434 (+ 420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:03:22.290438 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:03:22.290816 (+ 378us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:03:22.290967 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:22.290977 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:22.291816 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:22.291819 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:22.292466 (+ 647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:22.292476 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:22.292917 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:22.292919 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:22.292990 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:22.293606 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:03:22.293619 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:03:22.295788 (+ 2169us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.192.254:37079: Server oryx/127.1.192.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":126,"threads_started":1}
W20260809 02:03:22.296096 11954 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:03:22.289620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.194:37963 (local address 127.1.192.254:37079)
0809 02:03:22.289793 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0809 02:03:22.289798 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:03:22.289975 (+ 177us) server_negotiation.cc:407] Connection header received
0809 02:03:22.290139 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:03:22.290141 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:03:22.290224 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:03:22.290326 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:03:22.291137 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:22.291683 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:22.292589 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:22.292812 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:22.292892 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:22.295968 (+ 3076us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.192.194:37963: BlockingRecv error: recv got EOF from 127.1.192.194:37963 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 09 02:03:22 dist-test-slave-ss4n krb5kdc[10795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.192.195@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:03:22 dist-test-slave-ss4n krb5kdc[10795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.192.195@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:03:22.496824 11956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:22.490463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:37079 (local address 127.1.192.195:54257)
0809 02:03:22.490838 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:22.490853 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 02:03:22.490960 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:03:22.491275 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:03:22.491279 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:03:22.491470 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:03:22.491569 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:22.491575 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:22.492653 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:22.492655 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:22.493213 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:22.493222 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:22.493762 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:22.493765 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:22.493833 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:22.494456 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:03:22.494477 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:03:22.496638 (+ 2161us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.192.254:37079: Server oryx/127.1.192.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":179,"threads_started":1}
W20260809 02:03:22.497114 11954 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:03:22.490671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.195:54257 (local address 127.1.192.254:37079)
0809 02:03:22.490852 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 02:03:22.490857 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:03:22.490940 (+ 83us) server_negotiation.cc:407] Connection header received
0809 02:03:22.491065 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:03:22.491072 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:03:22.491122 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:03:22.491205 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:03:22.491715 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:22.492532 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:22.493351 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:22.493609 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:22.493723 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:22.496806 (+ 3083us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.192.195:54257: BlockingRecv error: recv got EOF from 127.1.192.195:54257 (error 108)
Metrics: {"server-negotiator.queue_time_us":98}
Aug 09 02:03:23 dist-test-slave-ss4n krb5kdc[10795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.192.193@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:03:23 dist-test-slave-ss4n krb5kdc[10795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.192.193@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:03:23.099927 11957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:23.094128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:37079 (local address 127.1.192.193:43983)
0809 02:03:23.094479 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:23.094496 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 02:03:23.094624 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:03:23.094859 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:03:23.094861 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:03:23.095098 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:03:23.095202 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:23.095209 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:23.096049 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:23.096052 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:23.096762 (+ 710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:23.096774 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:23.097249 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:23.097252 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:23.097321 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:23.097889 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:03:23.097901 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:03:23.099787 (+ 1886us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.192.254:37079: Server oryx/127.1.192.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":99,"threads_started":1}
W20260809 02:03:23.100093 11958 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:03:23.094184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.193:43983 (local address 127.1.192.254:37079)
0809 02:03:23.094439 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0809 02:03:23.094442 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:03:23.094561 (+ 119us) server_negotiation.cc:407] Connection header received
0809 02:03:23.094698 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:03:23.094699 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:03:23.094740 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:03:23.094819 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:03:23.095323 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:23.095916 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:23.096893 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:23.097138 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:23.097222 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:23.099937 (+ 2715us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.192.193:43983: BlockingRecv error: recv got EOF from 127.1.192.193:43983 (error 108)
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":74,"threads_started":1}
Aug 09 02:03:23 dist-test-slave-ss4n krb5kdc[10795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.192.194@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:03:23 dist-test-slave-ss4n krb5kdc[10795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.192.194@KRBTEST.COM for oryx/127.1.192.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:03:23.303295 11959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:23.297222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:37079 (local address 127.1.192.194:58135)
0809 02:03:23.297731 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:23.297751 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 02:03:23.297904 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:03:23.298206 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:03:23.298209 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:03:23.298426 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:03:23.298520 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:23.298526 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:23.299302 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:23.299306 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:23.299884 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:23.299892 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:23.300388 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:23.300392 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:23.300478 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:23.301091 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:03:23.301103 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:03:23.303133 (+ 2030us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.192.254:37079: Server oryx/127.1.192.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":209,"threads_started":1}
W20260809 02:03:23.303416 11958 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:03:23.297451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.194:58135 (local address 127.1.192.254:37079)
0809 02:03:23.297638 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0809 02:03:23.297644 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:03:23.297843 (+ 199us) server_negotiation.cc:407] Connection header received
0809 02:03:23.298016 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:03:23.298021 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:03:23.298079 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:03:23.298184 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:03:23.298639 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:23.299184 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:23.300016 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:23.300257 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:23.300370 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:23.303306 (+ 2936us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.192.194:58135: BlockingRecv error: recv got EOF from 127.1.192.194:58135 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
I20260809 02:03:23.381045 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 11433
I20260809 02:03:23.390323 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 11603
I20260809 02:03:23.397598 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 11750
I20260809 02:03:23.407356 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 11353
2026-08-09T02:03:23Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (18260 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-09T02:03:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:03:23Z Disabled control of system clock
I20260809 02:03:23.444521 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786240941968412-1795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786240941968412-1795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786240941968412-1795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.192.254:33511
--webserver_interface=127.1.192.254
--webserver_port=0
--builtin_ntp_servers=127.1.192.212:39249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.192.254:33511
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260809 02:03:23.565819 11963 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:03:23.566148 11963 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:03:23.566188 11963 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:03:23.570920 11963 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:03:23.570990 11963 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:03:23.571008 11963 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:03:23.571027 11963 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:03:23.571049 11963 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:03:23.576112 11963 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:39249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786240941968412-1795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786240941968412-1795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.192.254:33511
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.192.254:33511
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.192.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11963
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786240941968412-1795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:03:23.577199 11963 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:03:23.578650 11963 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:03:23.587899 11972 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:03:23.587913 11970 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:03:23.587983 11963 server_base.cc:1034] running on GCE node
W20260809 02:03:23.588107 11969 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:03:23.588874 11963 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:03:23.590119 11963 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:03:23.591317 11963 hybrid_clock.cc:648] HybridClock initialized: now 1786241003591260 us; error 72 us; skew 500 ppm
I20260809 02:03:23.593292 11963 webserver.cc:460] Webserver started at http://127.1.192.254:41959/ using document root <none> and password file <none>
I20260809 02:03:23.593968 11963 fs_manager.cc:359] Metadata directory not provided
I20260809 02:03:23.594082 11963 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:03:23.594321 11963 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:03:23.596580 11963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786240941968412-1795-0/minicluster-data/master-0/data/instance:
uuid: "d55d28aef06e418fbc2cd1ba84193325"
format_stamp: "Formatted at 2026-08-09 02:03:23 on dist-test-slave-ss4n"
server_key: "bf79bd51561208065e5a4355a0978a52"
server_key_iv: "7b8a67b2d32c6c7ab295e476eb779823"
server_key_version: "encryptionkey@0"
I20260809 02:03:23.597194 11963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786240941968412-1795-0/minicluster-data/master-0/wal/instance:
uuid: "d55d28aef06e418fbc2cd1ba84193325"
format_stamp: "Formatted at 2026-08-09 02:03:23 on dist-test-slave-ss4n"
server_key: "bf79bd51561208065e5a4355a0978a52"
server_key_iv: "7b8a67b2d32c6c7ab295e476eb779823"
server_key_version: "encryptionkey@0"
I20260809 02:03:23.600956 11963 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260809 02:03:23.603703 11978 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:03:23.605015 11963 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 02:03:23.605201 11963 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786240941968412-1795-0/minicluster-data/master-0/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786240941968412-1795-0/minicluster-data/master-0/wal
uuid: "d55d28aef06e418fbc2cd1ba84193325"
format_stamp: "Formatted at 2026-08-09 02:03:23 on dist-test-slave-ss4n"
server_key: "bf79bd51561208065e5a4355a0978a52"
server_key_iv: "7b8a67b2d32c6c7ab295e476eb779823"
server_key_version: "encryptionkey@0"
I20260809 02:03:23.605327 11963 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786240941968412-1795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786240941968412-1795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786240941968412-1795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:03:23.617002 11963 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:03:23.617836 11963 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:03:23.618059 11963 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:03:23.625690 11963 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.254:33511
I20260809 02:03:23.625710 12035 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.254:33511 every 8 connection(s)
I20260809 02:03:23.626696 11963 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
I20260809 02:03:23.629622 12036 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:03:23.631928 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 11963
I20260809 02:03:23.632036 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786240941968412-1795-0/minicluster-data/master-0/wal/instance
I20260809 02:03:23.632423 1795 external_mini_cluster.cc:1428] Setting key 9553977b7c38222c7470697f8abda078
I20260809 02:03:23.636121 12036 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d55d28aef06e418fbc2cd1ba84193325: Bootstrap starting.
I20260809 02:03:23.638973 12036 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d55d28aef06e418fbc2cd1ba84193325: Neither blocks nor log segments found. Creating new log.
I20260809 02:03:23.639885 12036 log.cc:824] T 00000000000000000000000000000000 P d55d28aef06e418fbc2cd1ba84193325: Log is configured to *not* fsync() on all Append() calls
I20260809 02:03:23.642333 12036 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d55d28aef06e418fbc2cd1ba84193325: No bootstrap required, opened a new log
I20260809 02:03:23.643420 12040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:23.634020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54438 (local address 127.1.192.254:33511)
0809 02:03:23.634770 (+ 750us) server_negotiation.cc:206] Beginning negotiation
0809 02:03:23.634791 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0809 02:03:23.634840 (+ 49us) server_negotiation.cc:407] Connection header received
0809 02:03:23.635670 (+ 830us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:03:23.635695 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:03:23.636191 (+ 496us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:03:23.636567 (+ 376us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:03:23.637027 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:23.638514 (+ 1487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:23.639583 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:23.640294 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:23.640441 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:23.641534 (+ 1093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:03:23.641541 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:03:23.641608 (+ 67us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:03:23.642009 (+ 401us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:03:23.642080 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:03:23.642104 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:03:23.642232 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0809 02:03:23.642391 (+ 159us) server_negotiation.cc:299] Negotiation successful
0809 02:03:23.642692 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":132,"threads_started":1}
I20260809 02:03:23.646462 12036 raft_consensus.cc:348] T 00000000000000000000000000000000 P d55d28aef06e418fbc2cd1ba84193325 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d55d28aef06e418fbc2cd1ba84193325" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 33511 } }
I20260809 02:03:23.646736 12036 raft_consensus.cc:374] T 00000000000000000000000000000000 P d55d28aef06e418fbc2cd1ba84193325 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:03:23.646823 12036 raft_consensus.cc:729] T 00000000000000000000000000000000 P d55d28aef06e418fbc2cd1ba84193325 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d55d28aef06e418fbc2cd1ba84193325, State: Initialized, Role: FOLLOWER
I20260809 02:03:23.647256 12036 consensus_queue.cc:260] T 00000000000000000000000000000000 P d55d28aef06e418fbc2cd1ba84193325 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d55d28aef06e418fbc2cd1ba84193325" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 33511 } }
I20260809 02:03:23.647413 12036 raft_consensus.cc:388] T 00000000000000000000000000000000 P d55d28aef06e418fbc2cd1ba84193325 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:03:23.647501 12036 raft_consensus.cc:482] T 00000000000000000000000000000000 P d55d28aef06e418fbc2cd1ba84193325 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:03:23.647629 12036 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d55d28aef06e418fbc2cd1ba84193325 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:03:23.648653 12036 raft_consensus.cc:504] T 00000000000000000000000000000000 P d55d28aef06e418fbc2cd1ba84193325 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d55d28aef06e418fbc2cd1ba84193325" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 33511 } }
I20260809 02:03:23.649060 12036 leader_election.cc:302] T 00000000000000000000000000000000 P d55d28aef06e418fbc2cd1ba84193325 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d55d28aef06e418fbc2cd1ba84193325; no voters:
I20260809 02:03:23.649395 12036 leader_election.cc:288] T 00000000000000000000000000000000 P d55d28aef06e418fbc2cd1ba84193325 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:03:23.649677 12043 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d55d28aef06e418fbc2cd1ba84193325 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:03:23.649957 12043 raft_consensus.cc:686] T 00000000000000000000000000000000 P d55d28aef06e418fbc2cd1ba84193325 [term 1 LEADER]: Becoming Leader. State: Replica: d55d28aef06e418fbc2cd1ba84193325, State: Running, Role: LEADER
I20260809 02:03:23.650277 12043 consensus_queue.cc:237] T 00000000000000000000000000000000 P d55d28aef06e418fbc2cd1ba84193325 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d55d28aef06e418fbc2cd1ba84193325" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 33511 } }
I20260809 02:03:23.650796 12036 sys_catalog.cc:564] T 00000000000000000000000000000000 P d55d28aef06e418fbc2cd1ba84193325 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:03:23.651928 12044 sys_catalog.cc:455] T 00000000000000000000000000000000 P d55d28aef06e418fbc2cd1ba84193325 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d55d28aef06e418fbc2cd1ba84193325" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d55d28aef06e418fbc2cd1ba84193325" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 33511 } } }
I20260809 02:03:23.652078 12044 sys_catalog.cc:458] T 00000000000000000000000000000000 P d55d28aef06e418fbc2cd1ba84193325 [sys.catalog]: This master's current role is: LEADER
I20260809 02:03:23.652390 12045 sys_catalog.cc:455] T 00000000000000000000000000000000 P d55d28aef06e418fbc2cd1ba84193325 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d55d28aef06e418fbc2cd1ba84193325. Latest consensus state: current_term: 1 leader_uuid: "d55d28aef06e418fbc2cd1ba84193325" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d55d28aef06e418fbc2cd1ba84193325" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 33511 } } }
I20260809 02:03:23.652526 12045 sys_catalog.cc:458] T 00000000000000000000000000000000 P d55d28aef06e418fbc2cd1ba84193325 [sys.catalog]: This master's current role is: LEADER
I20260809 02:03:23.652602 12053 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:03:23.655467 12053 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:03:23.661229 12053 catalog_manager.cc:1347] Generated new cluster ID: e90d0c6e7fa74c5492e1f20c4cbe2fa8
I20260809 02:03:23.661319 12053 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:03:23.683300 12053 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:03:23.684626 12053 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:03:23.692965 12053 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d55d28aef06e418fbc2cd1ba84193325: Generated new TSK 0
I20260809 02:03:23.694018 12053 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:03:23.697901 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786240941968412-1795-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786240941968412-1795-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786240941968412-1795-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786240941968412-1795-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.192.193:0
--local_ip_for_outbound_sockets=127.1.192.193
--webserver_interface=127.1.192.193
--webserver_port=0
--tserver_master_addrs=127.1.192.254:33511
--builtin_ntp_servers=127.1.192.212:39249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260809 02:03:23.821427 12064 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:03:23.821791 12064 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:03:23.821863 12064 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:03:23.826030 12064 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:03:23.826116 12064 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:03:23.826210 12064 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.192.193
I20260809 02:03:23.831003 12064 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:39249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786240941968412-1795-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786240941968412-1795-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.192.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786240941968412-1795-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.192.193
--webserver_port=0
--tserver_master_addrs=127.1.192.254:33511
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12064
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.192.193
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786240941968412-1795-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:03:23.832212 12064 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:03:23.833642 12064 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:03:23.844328 12069 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:03:23.844331 12070 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:03:23.844329 12072 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:03:23.844580 12064 server_base.cc:1034] running on GCE node
I20260809 02:03:23.845266 12064 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:03:23.846029 12064 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:03:23.847230 12064 hybrid_clock.cc:648] HybridClock initialized: now 1786241003847209 us; error 39 us; skew 500 ppm
I20260809 02:03:23.849308 12064 webserver.cc:460] Webserver started at http://127.1.192.193:37479/ using document root <none> and password file <none>
I20260809 02:03:23.849962 12064 fs_manager.cc:359] Metadata directory not provided
I20260809 02:03:23.850061 12064 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:03:23.850276 12064 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:03:23.852602 12064 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786240941968412-1795-0/minicluster-data/ts-0/data/instance:
uuid: "0c0fbaceaf474052b76e73744ec69783"
format_stamp: "Formatted at 2026-08-09 02:03:23 on dist-test-slave-ss4n"
server_key: "dfd1c8d1953b3919d2a6b2cd643b571d"
server_key_iv: "c0f4e9e17e79408828c6874c9260a410"
server_key_version: "encryptionkey@0"
I20260809 02:03:23.853180 12064 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786240941968412-1795-0/minicluster-data/ts-0/wal/instance:
uuid: "0c0fbaceaf474052b76e73744ec69783"
format_stamp: "Formatted at 2026-08-09 02:03:23 on dist-test-slave-ss4n"
server_key: "dfd1c8d1953b3919d2a6b2cd643b571d"
server_key_iv: "c0f4e9e17e79408828c6874c9260a410"
server_key_version: "encryptionkey@0"
I20260809 02:03:23.857203 12064 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260809 02:03:23.860005 12078 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:03:23.861456 12064 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 02:03:23.861703 12064 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786240941968412-1795-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786240941968412-1795-0/minicluster-data/ts-0/wal
uuid: "0c0fbaceaf474052b76e73744ec69783"
format_stamp: "Formatted at 2026-08-09 02:03:23 on dist-test-slave-ss4n"
server_key: "dfd1c8d1953b3919d2a6b2cd643b571d"
server_key_iv: "c0f4e9e17e79408828c6874c9260a410"
server_key_version: "encryptionkey@0"
I20260809 02:03:23.861832 12064 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786240941968412-1795-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786240941968412-1795-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786240941968412-1795-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:03:23.874758 12064 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:03:23.875554 12064 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:03:23.875761 12064 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:03:23.876371 12064 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:03:23.877344 12064 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:03:23.877418 12064 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:03:23.877550 12064 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:03:23.877599 12064 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:03:23.887166 12064 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.193:45761
I20260809 02:03:23.887189 12203 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.193:45761 every 8 connection(s)
I20260809 02:03:23.888123 12064 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786240941968412-1795-0/minicluster-data/ts-0/data/info.pb
I20260809 02:03:23.896031 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 12064
I20260809 02:03:23.896184 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786240941968412-1795-0/minicluster-data/ts-0/wal/instance
I20260809 02:03:23.896507 1795 external_mini_cluster.cc:1428] Setting key f5fbe2fbbf111333f88c98e74e117d37
I20260809 02:03:23.897789 12040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:23.890075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.193:48493 (local address 127.1.192.254:33511)
0809 02:03:23.890217 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0809 02:03:23.890221 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:03:23.891344 (+ 1123us) server_negotiation.cc:407] Connection header received
0809 02:03:23.892786 (+ 1442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:03:23.892790 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:03:23.892852 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:03:23.892942 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:03:23.893915 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:23.894476 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:23.895629 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:23.895879 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:23.895999 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:23.897242 (+ 1243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:03:23.897246 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:03:23.897249 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:03:23.897263 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:03:23.897291 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:03:23.897299 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:03:23.897378 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 02:03:23.897581 (+ 203us) server_negotiation.cc:299] Negotiation successful
0809 02:03:23.897646 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 02:03:23.898566 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786240941968412-1795-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786240941968412-1795-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786240941968412-1795-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786240941968412-1795-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.192.194:0
--local_ip_for_outbound_sockets=127.1.192.194
--webserver_interface=127.1.192.194
--webserver_port=0
--tserver_master_addrs=127.1.192.254:33511
--builtin_ntp_servers=127.1.192.212:39249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260809 02:03:23.898715 12206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:23.890471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:33511 (local address 127.1.192.193:48493)
0809 02:03:23.891175 (+ 704us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:23.891218 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 02:03:23.892569 (+ 1351us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:03:23.893128 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:03:23.893135 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:03:23.893217 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:03:23.893773 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:23.893787 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:23.894627 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:23.894630 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:23.895504 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:23.895513 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:23.896059 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:23.896063 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:23.896290 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:23.896963 (+ 673us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 02:03:23.896965 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:03:23.897007 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 02:03:23.897079 (+ 72us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:03:23.897393 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:03:23.897408 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:03:23.897414 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 02:03:23.897580 (+ 166us) client_negotiation.cc:240] Negotiation successful
0809 02:03:23.897871 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":120,"threads_started":1}
I20260809 02:03:23.900187 12204 heartbeater.cc:347] Connected to a master server at 127.1.192.254:33511
I20260809 02:03:23.900445 12204 heartbeater.cc:464] Registering TS with master...
I20260809 02:03:23.900985 12204 heartbeater.cc:511] Master 127.1.192.254:33511 requested a full tablet report, sending...
I20260809 02:03:23.902662 11998 ts_manager.cc:194] Registered new tserver with Master: 0c0fbaceaf474052b76e73744ec69783 (127.1.192.193:45761)
I20260809 02:03:23.903846 11998 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.192.193:48493
W20260809 02:03:24.025617 12207 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:03:24.025913 12207 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:03:24.025951 12207 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:03:24.030187 12207 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:03:24.030256 12207 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:03:24.030356 12207 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.192.194
I20260809 02:03:24.035384 12207 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:39249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786240941968412-1795-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786240941968412-1795-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.192.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786240941968412-1795-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.192.194
--webserver_port=0
--tserver_master_addrs=127.1.192.254:33511
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12207
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.192.194
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786240941968412-1795-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:03:24.036573 12207 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:03:24.038021 12207 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:03:24.047808 12213 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:03:24.047830 12215 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:03:24.047868 12212 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:03:24.048058 12207 server_base.cc:1034] running on GCE node
I20260809 02:03:24.048691 12207 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:03:24.049418 12207 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:03:24.050683 12207 hybrid_clock.cc:648] HybridClock initialized: now 1786241004050657 us; error 30 us; skew 500 ppm
I20260809 02:03:24.052872 12207 webserver.cc:460] Webserver started at http://127.1.192.194:34729/ using document root <none> and password file <none>
I20260809 02:03:24.053573 12207 fs_manager.cc:359] Metadata directory not provided
I20260809 02:03:24.053673 12207 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:03:24.053925 12207 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:03:24.056107 12207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786240941968412-1795-0/minicluster-data/ts-1/data/instance:
uuid: "ba5dbc9636a94953bced4bcf577e1e2e"
format_stamp: "Formatted at 2026-08-09 02:03:24 on dist-test-slave-ss4n"
server_key: "3540a3f72f46263c5cbbe2fa9902ff8f"
server_key_iv: "9b6f041eb44c3678254d51d78a3840ca"
server_key_version: "encryptionkey@0"
I20260809 02:03:24.056734 12207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786240941968412-1795-0/minicluster-data/ts-1/wal/instance:
uuid: "ba5dbc9636a94953bced4bcf577e1e2e"
format_stamp: "Formatted at 2026-08-09 02:03:24 on dist-test-slave-ss4n"
server_key: "3540a3f72f46263c5cbbe2fa9902ff8f"
server_key_iv: "9b6f041eb44c3678254d51d78a3840ca"
server_key_version: "encryptionkey@0"
I20260809 02:03:24.060799 12207 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260809 02:03:24.063520 12222 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:03:24.064705 12207 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 02:03:24.064841 12207 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786240941968412-1795-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786240941968412-1795-0/minicluster-data/ts-1/wal
uuid: "ba5dbc9636a94953bced4bcf577e1e2e"
format_stamp: "Formatted at 2026-08-09 02:03:24 on dist-test-slave-ss4n"
server_key: "3540a3f72f46263c5cbbe2fa9902ff8f"
server_key_iv: "9b6f041eb44c3678254d51d78a3840ca"
server_key_version: "encryptionkey@0"
I20260809 02:03:24.064985 12207 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786240941968412-1795-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786240941968412-1795-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786240941968412-1795-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:03:24.081475 12207 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:03:24.082294 12207 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:03:24.082510 12207 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:03:24.083132 12207 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:03:24.084111 12207 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:03:24.084192 12207 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:03:24.084263 12207 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:03:24.084316 12207 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:03:24.093734 12207 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.194:35277
I20260809 02:03:24.093761 12344 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.194:35277 every 8 connection(s)
I20260809 02:03:24.094790 12207 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786240941968412-1795-0/minicluster-data/ts-1/data/info.pb
I20260809 02:03:24.097625 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 12207
I20260809 02:03:24.097767 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786240941968412-1795-0/minicluster-data/ts-1/wal/instance
I20260809 02:03:24.098246 1795 external_mini_cluster.cc:1428] Setting key 1f6a89dd056c0c167691c8d0b328d5a5
I20260809 02:03:24.100749 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786240941968412-1795-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786240941968412-1795-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786240941968412-1795-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786240941968412-1795-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.192.195:0
--local_ip_for_outbound_sockets=127.1.192.195
--webserver_interface=127.1.192.195
--webserver_port=0
--tserver_master_addrs=127.1.192.254:33511
--builtin_ntp_servers=127.1.192.212:39249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260809 02:03:24.106388 12040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:24.096920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.194:36375 (local address 127.1.192.254:33511)
0809 02:03:24.097112 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0809 02:03:24.097116 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:03:24.098376 (+ 1260us) server_negotiation.cc:407] Connection header received
0809 02:03:24.099973 (+ 1597us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:03:24.099977 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:03:24.100038 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:03:24.100139 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:03:24.101115 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:24.102074 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:24.103910 (+ 1836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:24.104173 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:24.104268 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:24.105730 (+ 1462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:03:24.105736 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:03:24.105741 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:03:24.105767 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:03:24.105791 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:03:24.105801 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:03:24.105850 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 02:03:24.106073 (+ 223us) server_negotiation.cc:299] Negotiation successful
0809 02:03:24.106157 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 02:03:24.107201 12347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:24.097334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:33511 (local address 127.1.192.194:36375)
0809 02:03:24.098157 (+ 823us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:24.098209 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 02:03:24.099741 (+ 1532us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:03:24.100288 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:03:24.100295 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:03:24.100377 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:03:24.100857 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:24.100898 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:24.102239 (+ 1341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:24.102242 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:24.103761 (+ 1519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:24.103777 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:24.104351 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:24.104354 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:24.104572 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:24.105326 (+ 754us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 02:03:24.105328 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:03:24.105371 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 02:03:24.105439 (+ 68us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:03:24.105909 (+ 470us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:03:24.105917 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:03:24.105921 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 02:03:24.106019 (+ 98us) client_negotiation.cc:240] Negotiation successful
0809 02:03:24.106261 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":232,"threads_started":1}
I20260809 02:03:24.108493 12345 heartbeater.cc:347] Connected to a master server at 127.1.192.254:33511
I20260809 02:03:24.108762 12345 heartbeater.cc:464] Registering TS with master...
I20260809 02:03:24.109304 12345 heartbeater.cc:511] Master 127.1.192.254:33511 requested a full tablet report, sending...
I20260809 02:03:24.110414 11998 ts_manager.cc:194] Registered new tserver with Master: ba5dbc9636a94953bced4bcf577e1e2e (127.1.192.194:35277)
I20260809 02:03:24.110988 11998 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.192.194:36375
W20260809 02:03:24.221267 12348 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:03:24.221640 12348 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:03:24.221717 12348 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:03:24.226154 12348 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:03:24.226227 12348 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:03:24.226284 12348 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.192.195
I20260809 02:03:24.231176 12348 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:39249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786240941968412-1795-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786240941968412-1795-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.192.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786240941968412-1795-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.192.195
--webserver_port=0
--tserver_master_addrs=127.1.192.254:33511
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12348
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.192.195
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786240941968412-1795-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:03:24.232322 12348 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:03:24.233865 12348 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:03:24.244060 12357 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:03:24.244112 12354 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:03:24.244098 12355 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:03:24.244316 12348 server_base.cc:1034] running on GCE node
I20260809 02:03:24.244992 12348 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:03:24.245801 12348 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:03:24.247041 12348 hybrid_clock.cc:648] HybridClock initialized: now 1786241004247018 us; error 36 us; skew 500 ppm
I20260809 02:03:24.249333 12348 webserver.cc:460] Webserver started at http://127.1.192.195:41213/ using document root <none> and password file <none>
I20260809 02:03:24.250038 12348 fs_manager.cc:359] Metadata directory not provided
I20260809 02:03:24.250146 12348 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:03:24.250428 12348 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:03:24.252912 12348 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786240941968412-1795-0/minicluster-data/ts-2/data/instance:
uuid: "2612460156cc4adda59e83e7b959a8eb"
format_stamp: "Formatted at 2026-08-09 02:03:24 on dist-test-slave-ss4n"
server_key: "52792d233497a37ece933d642e33e818"
server_key_iv: "04dc05b9354288574505179070d28708"
server_key_version: "encryptionkey@0"
I20260809 02:03:24.253707 12348 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786240941968412-1795-0/minicluster-data/ts-2/wal/instance:
uuid: "2612460156cc4adda59e83e7b959a8eb"
format_stamp: "Formatted at 2026-08-09 02:03:24 on dist-test-slave-ss4n"
server_key: "52792d233497a37ece933d642e33e818"
server_key_iv: "04dc05b9354288574505179070d28708"
server_key_version: "encryptionkey@0"
I20260809 02:03:24.257817 12348 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260809 02:03:24.260628 12363 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:03:24.262087 12348 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 02:03:24.262264 12348 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786240941968412-1795-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786240941968412-1795-0/minicluster-data/ts-2/wal
uuid: "2612460156cc4adda59e83e7b959a8eb"
format_stamp: "Formatted at 2026-08-09 02:03:24 on dist-test-slave-ss4n"
server_key: "52792d233497a37ece933d642e33e818"
server_key_iv: "04dc05b9354288574505179070d28708"
server_key_version: "encryptionkey@0"
I20260809 02:03:24.262401 12348 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786240941968412-1795-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786240941968412-1795-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786240941968412-1795-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:03:24.284924 12348 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:03:24.285768 12348 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:03:24.286000 12348 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:03:24.286675 12348 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:03:24.287730 12348 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:03:24.287807 12348 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:03:24.287889 12348 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:03:24.287935 12348 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:03:24.297782 12348 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.195:36575
I20260809 02:03:24.297868 12487 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.195:36575 every 8 connection(s)
I20260809 02:03:24.298853 12348 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786240941968412-1795-0/minicluster-data/ts-2/data/info.pb
I20260809 02:03:24.299996 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 12348
I20260809 02:03:24.300123 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786240941968412-1795-0/minicluster-data/ts-2/wal/instance
I20260809 02:03:24.300451 1795 external_mini_cluster.cc:1428] Setting key 785307091ebd8954e4b9174e0419c232
I20260809 02:03:24.309758 12040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:24.301184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.195:52555 (local address 127.1.192.254:33511)
0809 02:03:24.301344 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0809 02:03:24.301348 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:03:24.302677 (+ 1329us) server_negotiation.cc:407] Connection header received
0809 02:03:24.304279 (+ 1602us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:03:24.304283 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:03:24.304343 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:03:24.304432 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:03:24.305397 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:24.306069 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:24.307390 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:24.307667 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:24.307795 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:24.309145 (+ 1350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:03:24.309149 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:03:24.309153 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:03:24.309171 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:03:24.309197 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:03:24.309206 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:03:24.309296 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0809 02:03:24.309477 (+ 181us) server_negotiation.cc:299] Negotiation successful
0809 02:03:24.309591 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 02:03:24.310705 12490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:24.301710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:33511 (local address 127.1.192.195:52555)
0809 02:03:24.302523 (+ 813us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:24.302572 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 02:03:24.304056 (+ 1484us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:03:24.304614 (+ 558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:03:24.304622 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:03:24.304715 (+ 93us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:03:24.305243 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:24.305257 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:24.306214 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:24.306227 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:24.307246 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:24.307257 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:24.307841 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:24.307846 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:24.308032 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:24.308802 (+ 770us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 02:03:24.308806 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:03:24.308867 (+ 61us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 02:03:24.308941 (+ 74us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:03:24.309297 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:03:24.309311 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:03:24.309318 (+ 7us) client_negotiation.cc:769] Sending connection context
0809 02:03:24.309464 (+ 146us) client_negotiation.cc:240] Negotiation successful
0809 02:03:24.309822 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":169,"threads_started":1}
I20260809 02:03:24.311973 12488 heartbeater.cc:347] Connected to a master server at 127.1.192.254:33511
I20260809 02:03:24.312220 12488 heartbeater.cc:464] Registering TS with master...
I20260809 02:03:24.312737 12488 heartbeater.cc:511] Master 127.1.192.254:33511 requested a full tablet report, sending...
I20260809 02:03:24.314221 11997 ts_manager.cc:194] Registered new tserver with Master: 2612460156cc4adda59e83e7b959a8eb (127.1.192.195:36575)
I20260809 02:03:24.315042 11997 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.192.195:52555
I20260809 02:03:24.315327 1795 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 02:03:24.318809 12040 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:03:24.317859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54444 (local address 127.1.192.254:33511)
0809 02:03:24.318007 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0809 02:03:24.318011 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:03:24.318196 (+ 185us) server_negotiation.cc:407] Connection header received
0809 02:03:24.318293 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:03:24.318295 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:03:24.318339 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:03:24.318416 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:03:24.318640 (+ 224us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54444: BlockingRecv error: recv got EOF from 127.0.0.1:54444 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 02:03:24.319355 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 12064
I20260809 02:03:24.331010 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 12207
I20260809 02:03:24.343125 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 12348
I20260809 02:03:24.349851 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 11963
2026-08-09T02:03:24Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (940 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-09T02:03:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:03:24Z Disabled control of system clock
I20260809 02:03:24.384136 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786240941968412-1795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786240941968412-1795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786240941968412-1795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.192.254:42613
--webserver_interface=127.1.192.254
--webserver_port=0
--builtin_ntp_servers=127.1.192.212:44139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.192.254:42613
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 02:03:24.502439 12501 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:03:24.502741 12501 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:03:24.502782 12501 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:03:24.506903 12501 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:03:24.507006 12501 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:03:24.507023 12501 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:03:24.507040 12501 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:03:24.507056 12501 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:03:24.512073 12501 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:44139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786240941968412-1795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786240941968412-1795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.192.254:42613
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.192.254:42613
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.192.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12501
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786240941968412-1795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:03:24.513164 12501 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:03:24.514590 12501 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:03:24.523284 12507 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:03:24.523277 12506 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:03:24.523276 12509 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:03:24.523680 12501 server_base.cc:1034] running on GCE node
I20260809 02:03:24.524209 12501 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:03:24.525359 12501 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:03:24.526568 12501 hybrid_clock.cc:648] HybridClock initialized: now 1786241004526519 us; error 29 us; skew 500 ppm
I20260809 02:03:24.528553 12501 webserver.cc:460] Webserver started at http://127.1.192.254:42127/ using document root <none> and password file <none>
I20260809 02:03:24.529168 12501 fs_manager.cc:359] Metadata directory not provided
I20260809 02:03:24.529274 12501 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:03:24.529594 12501 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:03:24.531757 12501 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786240941968412-1795-0/minicluster-data/master-0/data/instance:
uuid: "8580c2e507074db3914aa747fa80b10a"
format_stamp: "Formatted at 2026-08-09 02:03:24 on dist-test-slave-ss4n"
server_key: "cc173f97919e65aa6777b69414c6c87b"
server_key_iv: "476a666d8814275bd5f3fd8ee1a95f08"
server_key_version: "encryptionkey@0"
I20260809 02:03:24.532378 12501 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786240941968412-1795-0/minicluster-data/master-0/wal/instance:
uuid: "8580c2e507074db3914aa747fa80b10a"
format_stamp: "Formatted at 2026-08-09 02:03:24 on dist-test-slave-ss4n"
server_key: "cc173f97919e65aa6777b69414c6c87b"
server_key_iv: "476a666d8814275bd5f3fd8ee1a95f08"
server_key_version: "encryptionkey@0"
I20260809 02:03:24.536271 12501 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 02:03:24.539057 12516 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:03:24.540370 12501 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 02:03:24.540566 12501 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786240941968412-1795-0/minicluster-data/master-0/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786240941968412-1795-0/minicluster-data/master-0/wal
uuid: "8580c2e507074db3914aa747fa80b10a"
format_stamp: "Formatted at 2026-08-09 02:03:24 on dist-test-slave-ss4n"
server_key: "cc173f97919e65aa6777b69414c6c87b"
server_key_iv: "476a666d8814275bd5f3fd8ee1a95f08"
server_key_version: "encryptionkey@0"
I20260809 02:03:24.540710 12501 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786240941968412-1795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786240941968412-1795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786240941968412-1795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:03:24.545641 12501 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:03:24.546321 12501 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:03:24.546528 12501 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:03:24.553709 12501 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.254:42613
I20260809 02:03:24.553728 12574 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.254:42613 every 8 connection(s)
I20260809 02:03:24.554713 12501 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
I20260809 02:03:24.558281 12575 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:03:24.560959 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 12501
I20260809 02:03:24.561115 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786240941968412-1795-0/minicluster-data/master-0/wal/instance
I20260809 02:03:24.561429 1795 external_mini_cluster.cc:1428] Setting key e63d15bdbbb44f804d5d9cbe3eece251
I20260809 02:03:24.565618 12575 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8580c2e507074db3914aa747fa80b10a: Bootstrap starting.
I20260809 02:03:24.567977 12580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:24.563243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39494 (local address 127.1.192.254:42613)
0809 02:03:24.563970 (+ 727us) server_negotiation.cc:206] Beginning negotiation
0809 02:03:24.563987 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 02:03:24.564031 (+ 44us) server_negotiation.cc:407] Connection header received
0809 02:03:24.564815 (+ 784us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:03:24.564833 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:03:24.565192 (+ 359us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:03:24.565646 (+ 454us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:03:24.566650 (+ 1004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:03:24.566655 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:03:24.566666 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:03:24.566677 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:03:24.566732 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:03:24.566747 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:03:24.566866 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0809 02:03:24.566989 (+ 123us) server_negotiation.cc:299] Negotiation successful
0809 02:03:24.567267 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":369,"thread_start_us":166,"threads_started":1}
I20260809 02:03:24.568523 12575 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8580c2e507074db3914aa747fa80b10a: Neither blocks nor log segments found. Creating new log.
I20260809 02:03:24.569446 12575 log.cc:824] T 00000000000000000000000000000000 P 8580c2e507074db3914aa747fa80b10a: Log is configured to *not* fsync() on all Append() calls
I20260809 02:03:24.571589 12575 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8580c2e507074db3914aa747fa80b10a: No bootstrap required, opened a new log
I20260809 02:03:24.575470 12575 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8580c2e507074db3914aa747fa80b10a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8580c2e507074db3914aa747fa80b10a" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 42613 } }
I20260809 02:03:24.575753 12575 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8580c2e507074db3914aa747fa80b10a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:03:24.575819 12575 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8580c2e507074db3914aa747fa80b10a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8580c2e507074db3914aa747fa80b10a, State: Initialized, Role: FOLLOWER
I20260809 02:03:24.576267 12575 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8580c2e507074db3914aa747fa80b10a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8580c2e507074db3914aa747fa80b10a" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 42613 } }
I20260809 02:03:24.576436 12575 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8580c2e507074db3914aa747fa80b10a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:03:24.576541 12575 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8580c2e507074db3914aa747fa80b10a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:03:24.576651 12575 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8580c2e507074db3914aa747fa80b10a [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:03:24.577804 12575 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8580c2e507074db3914aa747fa80b10a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8580c2e507074db3914aa747fa80b10a" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 42613 } }
I20260809 02:03:24.578176 12575 leader_election.cc:302] T 00000000000000000000000000000000 P 8580c2e507074db3914aa747fa80b10a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8580c2e507074db3914aa747fa80b10a; no voters:
I20260809 02:03:24.578573 12575 leader_election.cc:288] T 00000000000000000000000000000000 P 8580c2e507074db3914aa747fa80b10a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:03:24.578655 12582 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8580c2e507074db3914aa747fa80b10a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:03:24.578908 12582 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8580c2e507074db3914aa747fa80b10a [term 1 LEADER]: Becoming Leader. State: Replica: 8580c2e507074db3914aa747fa80b10a, State: Running, Role: LEADER
I20260809 02:03:24.579239 12582 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8580c2e507074db3914aa747fa80b10a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8580c2e507074db3914aa747fa80b10a" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 42613 } }
I20260809 02:03:24.580030 12575 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8580c2e507074db3914aa747fa80b10a [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:03:24.581115 12583 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8580c2e507074db3914aa747fa80b10a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8580c2e507074db3914aa747fa80b10a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8580c2e507074db3914aa747fa80b10a" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 42613 } } }
I20260809 02:03:24.581243 12583 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8580c2e507074db3914aa747fa80b10a [sys.catalog]: This master's current role is: LEADER
I20260809 02:03:24.581166 12584 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8580c2e507074db3914aa747fa80b10a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8580c2e507074db3914aa747fa80b10a. Latest consensus state: current_term: 1 leader_uuid: "8580c2e507074db3914aa747fa80b10a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8580c2e507074db3914aa747fa80b10a" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 42613 } } }
I20260809 02:03:24.581295 12584 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8580c2e507074db3914aa747fa80b10a [sys.catalog]: This master's current role is: LEADER
I20260809 02:03:24.581820 12592 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:03:24.584980 12592 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:03:24.590909 12592 catalog_manager.cc:1347] Generated new cluster ID: bf83d16bd52d4fff9c5c1ebf256b0059
I20260809 02:03:24.591028 12592 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:03:24.607702 12592 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:03:24.608106 12592 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:03:24.623265 12592 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8580c2e507074db3914aa747fa80b10a: Generated new TSK 0
I20260809 02:03:24.624292 12592 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:03:24.636153 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786240941968412-1795-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786240941968412-1795-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786240941968412-1795-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786240941968412-1795-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.192.193:0
--local_ip_for_outbound_sockets=127.1.192.193
--webserver_interface=127.1.192.193
--webserver_port=0
--tserver_master_addrs=127.1.192.254:42613
--builtin_ntp_servers=127.1.192.212:44139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 02:03:24.757329 12602 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:03:24.757700 12602 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:03:24.757746 12602 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:03:24.761826 12602 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:03:24.761894 12602 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:03:24.761967 12602 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.192.193
I20260809 02:03:24.766917 12602 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:44139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786240941968412-1795-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786240941968412-1795-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.192.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786240941968412-1795-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.192.193
--webserver_port=0
--tserver_master_addrs=127.1.192.254:42613
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12602
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.192.193
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786240941968412-1795-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:03:24.768046 12602 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:03:24.769415 12602 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:03:24.779403 12608 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:03:24.779394 12610 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:03:24.779403 12607 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:03:24.779672 12602 server_base.cc:1034] running on GCE node
I20260809 02:03:24.780299 12602 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:03:24.781005 12602 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:03:24.782224 12602 hybrid_clock.cc:648] HybridClock initialized: now 1786241004782173 us; error 64 us; skew 500 ppm
I20260809 02:03:24.784338 12602 webserver.cc:460] Webserver started at http://127.1.192.193:41471/ using document root <none> and password file <none>
I20260809 02:03:24.784945 12602 fs_manager.cc:359] Metadata directory not provided
I20260809 02:03:24.785015 12602 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:03:24.785234 12602 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:03:24.787544 12602 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786240941968412-1795-0/minicluster-data/ts-0/data/instance:
uuid: "dc12e15df94d44f98e3aed34eb31bae9"
format_stamp: "Formatted at 2026-08-09 02:03:24 on dist-test-slave-ss4n"
server_key: "07f4e2f3059fb1932ff2991911e54b09"
server_key_iv: "e2c89d3f6c64120c6bc9402d6a29997e"
server_key_version: "encryptionkey@0"
I20260809 02:03:24.788123 12602 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786240941968412-1795-0/minicluster-data/ts-0/wal/instance:
uuid: "dc12e15df94d44f98e3aed34eb31bae9"
format_stamp: "Formatted at 2026-08-09 02:03:24 on dist-test-slave-ss4n"
server_key: "07f4e2f3059fb1932ff2991911e54b09"
server_key_iv: "e2c89d3f6c64120c6bc9402d6a29997e"
server_key_version: "encryptionkey@0"
I20260809 02:03:24.792210 12602 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.002s
I20260809 02:03:24.795192 12616 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:03:24.796631 12602 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 02:03:24.796736 12602 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786240941968412-1795-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786240941968412-1795-0/minicluster-data/ts-0/wal
uuid: "dc12e15df94d44f98e3aed34eb31bae9"
format_stamp: "Formatted at 2026-08-09 02:03:24 on dist-test-slave-ss4n"
server_key: "07f4e2f3059fb1932ff2991911e54b09"
server_key_iv: "e2c89d3f6c64120c6bc9402d6a29997e"
server_key_version: "encryptionkey@0"
I20260809 02:03:24.796895 12602 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786240941968412-1795-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786240941968412-1795-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786240941968412-1795-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:03:24.801666 12602 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:03:24.802304 12602 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:03:24.802496 12602 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:03:24.803040 12602 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:03:24.804005 12602 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:03:24.804052 12602 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:03:24.804090 12602 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:03:24.804142 12602 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:03:24.813679 12602 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.193:46015
I20260809 02:03:24.813699 12740 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.193:46015 every 8 connection(s)
I20260809 02:03:24.814653 12602 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786240941968412-1795-0/minicluster-data/ts-0/data/info.pb
I20260809 02:03:24.821405 12580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:24.816687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.193:35365 (local address 127.1.192.254:42613)
0809 02:03:24.816861 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0809 02:03:24.816865 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:03:24.818081 (+ 1216us) server_negotiation.cc:407] Connection header received
0809 02:03:24.819526 (+ 1445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:03:24.819529 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:03:24.819573 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:03:24.819662 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:03:24.820884 (+ 1222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:03:24.820888 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:03:24.820890 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:03:24.820903 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:03:24.820926 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:03:24.820935 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:03:24.821055 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0809 02:03:24.821225 (+ 170us) server_negotiation.cc:299] Negotiation successful
0809 02:03:24.821274 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 02:03:24.822445 12744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:24.817056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:42613 (local address 127.1.192.193:35365)
0809 02:03:24.817857 (+ 801us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:24.817915 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0809 02:03:24.819284 (+ 1369us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:03:24.819832 (+ 548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:03:24.819839 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:03:24.819931 (+ 92us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:03:24.820577 (+ 646us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 02:03:24.820579 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:03:24.820630 (+ 51us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 02:03:24.820707 (+ 77us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:03:24.821038 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:03:24.821049 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:03:24.821054 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 02:03:24.821204 (+ 150us) client_negotiation.cc:240] Negotiation successful
0809 02:03:24.821410 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":146,"threads_started":1}
I20260809 02:03:24.824007 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 12602
I20260809 02:03:24.824157 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786240941968412-1795-0/minicluster-data/ts-0/wal/instance
I20260809 02:03:24.824123 12742 heartbeater.cc:347] Connected to a master server at 127.1.192.254:42613
I20260809 02:03:24.824383 12742 heartbeater.cc:464] Registering TS with master...
I20260809 02:03:24.824508 1795 external_mini_cluster.cc:1428] Setting key 2ddec8d92fb59bb905d8b3333bcf6123
I20260809 02:03:24.824873 12742 heartbeater.cc:511] Master 127.1.192.254:42613 requested a full tablet report, sending...
I20260809 02:03:24.826660 12536 ts_manager.cc:194] Registered new tserver with Master: dc12e15df94d44f98e3aed34eb31bae9 (127.1.192.193:46015)
I20260809 02:03:24.826969 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786240941968412-1795-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786240941968412-1795-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786240941968412-1795-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786240941968412-1795-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.192.194:0
--local_ip_for_outbound_sockets=127.1.192.194
--webserver_interface=127.1.192.194
--webserver_port=0
--tserver_master_addrs=127.1.192.254:42613
--builtin_ntp_servers=127.1.192.212:44139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 02:03:24.942677 12745 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:03:24.942968 12745 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:03:24.943011 12745 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:03:24.948282 12745 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:03:24.948369 12745 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:03:24.948475 12745 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.192.194
I20260809 02:03:24.953732 12745 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:44139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786240941968412-1795-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786240941968412-1795-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.192.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786240941968412-1795-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.192.194
--webserver_port=0
--tserver_master_addrs=127.1.192.254:42613
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12745
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.192.194
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786240941968412-1795-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:03:24.954934 12745 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:03:24.956388 12745 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:03:24.966606 12750 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:03:24.966566 12753 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:03:24.966673 12751 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:03:24.966982 12745 server_base.cc:1034] running on GCE node
I20260809 02:03:24.967386 12745 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:03:24.968282 12745 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:03:24.969456 12745 hybrid_clock.cc:648] HybridClock initialized: now 1786241004969439 us; error 53 us; skew 500 ppm
I20260809 02:03:24.971534 12745 webserver.cc:460] Webserver started at http://127.1.192.194:37443/ using document root <none> and password file <none>
I20260809 02:03:24.972121 12745 fs_manager.cc:359] Metadata directory not provided
I20260809 02:03:24.972211 12745 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:03:24.972440 12745 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:03:24.974745 12745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786240941968412-1795-0/minicluster-data/ts-1/data/instance:
uuid: "5bb8dcf4c1804dc2a5bf8fcf51df4aeb"
format_stamp: "Formatted at 2026-08-09 02:03:24 on dist-test-slave-ss4n"
server_key: "e5b8094c4fe0c17d42804da2198ba800"
server_key_iv: "8bea7bb43754bc4b018ce0bd8449204a"
server_key_version: "encryptionkey@0"
I20260809 02:03:24.975419 12745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786240941968412-1795-0/minicluster-data/ts-1/wal/instance:
uuid: "5bb8dcf4c1804dc2a5bf8fcf51df4aeb"
format_stamp: "Formatted at 2026-08-09 02:03:24 on dist-test-slave-ss4n"
server_key: "e5b8094c4fe0c17d42804da2198ba800"
server_key_iv: "8bea7bb43754bc4b018ce0bd8449204a"
server_key_version: "encryptionkey@0"
I20260809 02:03:24.979661 12745 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 02:03:24.982784 12760 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:03:24.984102 12745 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 02:03:24.984201 12745 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786240941968412-1795-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786240941968412-1795-0/minicluster-data/ts-1/wal
uuid: "5bb8dcf4c1804dc2a5bf8fcf51df4aeb"
format_stamp: "Formatted at 2026-08-09 02:03:24 on dist-test-slave-ss4n"
server_key: "e5b8094c4fe0c17d42804da2198ba800"
server_key_iv: "8bea7bb43754bc4b018ce0bd8449204a"
server_key_version: "encryptionkey@0"
I20260809 02:03:24.984298 12745 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786240941968412-1795-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786240941968412-1795-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786240941968412-1795-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:03:24.989073 12745 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:03:24.989804 12745 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:03:24.989989 12745 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:03:24.990562 12745 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:03:24.991516 12745 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:03:24.991564 12745 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:03:24.991637 12745 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:03:24.991678 12745 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:03:25.000945 12745 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.194:45753
I20260809 02:03:25.001048 12882 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.194:45753 every 8 connection(s)
I20260809 02:03:25.001938 12745 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786240941968412-1795-0/minicluster-data/ts-1/data/info.pb
I20260809 02:03:25.004989 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 12745
I20260809 02:03:25.005143 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786240941968412-1795-0/minicluster-data/ts-1/wal/instance
I20260809 02:03:25.005550 1795 external_mini_cluster.cc:1428] Setting key cf92236665caeb5768aa678833a1822a
I20260809 02:03:25.008255 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786240941968412-1795-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786240941968412-1795-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786240941968412-1795-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786240941968412-1795-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.192.195:0
--local_ip_for_outbound_sockets=127.1.192.195
--webserver_interface=127.1.192.195
--webserver_port=0
--tserver_master_addrs=127.1.192.254:42613
--builtin_ntp_servers=127.1.192.212:44139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260809 02:03:25.009011 12580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:25.003993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.194:54281 (local address 127.1.192.254:42613)
0809 02:03:25.004165 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0809 02:03:25.004171 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:03:25.005517 (+ 1346us) server_negotiation.cc:407] Connection header received
0809 02:03:25.007100 (+ 1583us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:03:25.007103 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:03:25.007139 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:03:25.007216 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:03:25.008514 (+ 1298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:03:25.008517 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:03:25.008520 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:03:25.008533 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:03:25.008564 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:03:25.008573 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:03:25.008686 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 02:03:25.008827 (+ 141us) server_negotiation.cc:299] Negotiation successful
0809 02:03:25.008872 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 02:03:25.009994 12887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:25.004503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:42613 (local address 127.1.192.194:54281)
0809 02:03:25.005343 (+ 840us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:25.005384 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 02:03:25.006874 (+ 1490us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:03:25.007374 (+ 500us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:03:25.007381 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:03:25.007466 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:03:25.008188 (+ 722us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 02:03:25.008190 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:03:25.008231 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 02:03:25.008300 (+ 69us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:03:25.008671 (+ 371us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:03:25.008680 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:03:25.008686 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 02:03:25.008873 (+ 187us) client_negotiation.cc:240] Negotiation successful
0809 02:03:25.009115 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":175,"threads_started":1}
I20260809 02:03:25.011215 12885 heartbeater.cc:347] Connected to a master server at 127.1.192.254:42613
I20260809 02:03:25.011513 12885 heartbeater.cc:464] Registering TS with master...
I20260809 02:03:25.011986 12885 heartbeater.cc:511] Master 127.1.192.254:42613 requested a full tablet report, sending...
I20260809 02:03:25.013190 12536 ts_manager.cc:194] Registered new tserver with Master: 5bb8dcf4c1804dc2a5bf8fcf51df4aeb (127.1.192.194:45753)
W20260809 02:03:25.129364 12888 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:03:25.129768 12888 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:03:25.129812 12888 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:03:25.133953 12888 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:03:25.134009 12888 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:03:25.134065 12888 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.192.195
I20260809 02:03:25.138933 12888 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:44139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786240941968412-1795-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786240941968412-1795-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.192.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786240941968412-1795-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.192.195
--webserver_port=0
--tserver_master_addrs=127.1.192.254:42613
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12888
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.192.195
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786240941968412-1795-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:03:25.139991 12888 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:03:25.141355 12888 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:03:25.151409 12894 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:03:25.151448 12897 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:03:25.151464 12893 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:03:25.151865 12888 server_base.cc:1034] running on GCE node
I20260809 02:03:25.152313 12888 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:03:25.153072 12888 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:03:25.154256 12888 hybrid_clock.cc:648] HybridClock initialized: now 1786241005154230 us; error 40 us; skew 500 ppm
I20260809 02:03:25.156275 12888 webserver.cc:460] Webserver started at http://127.1.192.195:35513/ using document root <none> and password file <none>
I20260809 02:03:25.156922 12888 fs_manager.cc:359] Metadata directory not provided
I20260809 02:03:25.157001 12888 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:03:25.157235 12888 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:03:25.159446 12888 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786240941968412-1795-0/minicluster-data/ts-2/data/instance:
uuid: "e03d3cec21cb47bd9c9bfcc4e6a2f373"
format_stamp: "Formatted at 2026-08-09 02:03:25 on dist-test-slave-ss4n"
server_key: "e3bb5f9ebdb3a60a155e8e7be4b43816"
server_key_iv: "2c763e13a3a0391b81bacac89ac6199d"
server_key_version: "encryptionkey@0"
I20260809 02:03:25.160068 12888 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786240941968412-1795-0/minicluster-data/ts-2/wal/instance:
uuid: "e03d3cec21cb47bd9c9bfcc4e6a2f373"
format_stamp: "Formatted at 2026-08-09 02:03:25 on dist-test-slave-ss4n"
server_key: "e3bb5f9ebdb3a60a155e8e7be4b43816"
server_key_iv: "2c763e13a3a0391b81bacac89ac6199d"
server_key_version: "encryptionkey@0"
I20260809 02:03:25.164140 12888 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260809 02:03:25.166908 12904 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:03:25.168287 12888 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 02:03:25.168411 12888 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786240941968412-1795-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786240941968412-1795-0/minicluster-data/ts-2/wal
uuid: "e03d3cec21cb47bd9c9bfcc4e6a2f373"
format_stamp: "Formatted at 2026-08-09 02:03:25 on dist-test-slave-ss4n"
server_key: "e3bb5f9ebdb3a60a155e8e7be4b43816"
server_key_iv: "2c763e13a3a0391b81bacac89ac6199d"
server_key_version: "encryptionkey@0"
I20260809 02:03:25.168532 12888 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786240941968412-1795-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786240941968412-1795-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786240941968412-1795-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:03:25.173537 12888 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:03:25.174270 12888 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:03:25.174461 12888 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:03:25.175050 12888 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:03:25.176061 12888 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:03:25.176118 12888 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:03:25.176182 12888 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:03:25.176219 12888 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:03:25.185223 12888 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.195:33553
I20260809 02:03:25.185248 13023 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.195:33553 every 8 connection(s)
I20260809 02:03:25.186269 12888 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786240941968412-1795-0/minicluster-data/ts-2/data/info.pb
I20260809 02:03:25.187676 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 12888
I20260809 02:03:25.187788 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786240941968412-1795-0/minicluster-data/ts-2/wal/instance
I20260809 02:03:25.188109 1795 external_mini_cluster.cc:1428] Setting key c99175b497998c203f74a451ce9e123c
I20260809 02:03:25.192736 12580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:25.188362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.195:44331 (local address 127.1.192.254:42613)
0809 02:03:25.188566 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0809 02:03:25.188571 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:03:25.189824 (+ 1253us) server_negotiation.cc:407] Connection header received
0809 02:03:25.191113 (+ 1289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:03:25.191116 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:03:25.191153 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:03:25.191224 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:03:25.192333 (+ 1109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:03:25.192335 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:03:25.192337 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:03:25.192349 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:03:25.192369 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:03:25.192376 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:03:25.192450 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 02:03:25.192583 (+ 133us) server_negotiation.cc:299] Negotiation successful
0809 02:03:25.192621 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260809 02:03:25.193689 13027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:25.188896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:42613 (local address 127.1.192.195:44331)
0809 02:03:25.189660 (+ 764us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:25.189714 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 02:03:25.190901 (+ 1187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:03:25.191382 (+ 481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:03:25.191389 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:03:25.191474 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:03:25.192086 (+ 612us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 02:03:25.192088 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:03:25.192128 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 02:03:25.192196 (+ 68us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:03:25.192452 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:03:25.192458 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:03:25.192462 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 02:03:25.192592 (+ 130us) client_negotiation.cc:240] Negotiation successful
0809 02:03:25.192803 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":134,"threads_started":1}
I20260809 02:03:25.194880 13025 heartbeater.cc:347] Connected to a master server at 127.1.192.254:42613
I20260809 02:03:25.195096 13025 heartbeater.cc:464] Registering TS with master...
I20260809 02:03:25.195477 13025 heartbeater.cc:511] Master 127.1.192.254:42613 requested a full tablet report, sending...
I20260809 02:03:25.196576 12536 ts_manager.cc:194] Registered new tserver with Master: e03d3cec21cb47bd9c9bfcc4e6a2f373 (127.1.192.195:33553)
I20260809 02:03:25.202332 1795 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 02:03:25.205523 12580 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:03:25.204370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39498 (local address 127.1.192.254:42613)
0809 02:03:25.204637 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0809 02:03:25.204643 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:03:25.204771 (+ 128us) server_negotiation.cc:407] Connection header received
0809 02:03:25.204930 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:03:25.204933 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:03:25.204983 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:03:25.205084 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:03:25.205351 (+ 267us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39498: BlockingRecv error: recv got EOF from 127.0.0.1:39498 (error 108)
Metrics: {"server-negotiator.queue_time_us":168}
I20260809 02:03:25.205981 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 12602
I20260809 02:03:25.217101 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 12745
I20260809 02:03:25.224980 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 12888
I20260809 02:03:25.234833 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 12501
2026-08-09T02:03:25Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (887 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786240941968412-1795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:03:25 dist-test-slave-ss4n krb5kdc[13037](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:03:25 dist-test-slave-ss4n krb5kdc[13037](info): set up 2 sockets
Aug 09 02:03:25 dist-test-slave-ss4n krb5kdc[13037](info): commencing operation
krb5kdc: starting...
W20260809 02:03:26.337950 1795 mini_kdc.cc:121] Time spent starting KDC: real 1.077s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786240941968412-1795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786240941968412-1795-0/krb5kdc/test-user.keytab.
Aug 09 02:03:26 dist-test-slave-ss4n krb5kdc[13037](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241006, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:03:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:03:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.192.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.192.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.192.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab.
Entry for principal kudu/127.1.192.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab.
I20260809 02:03:26.485432 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786240941968412-1795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786240941968412-1795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786240941968412-1795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.192.254:44101
--webserver_interface=127.1.192.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab
--principal=kudu/127.1.192.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:45343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.192.254:44101
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:03:26.599090 13052 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:03:26.599390 13052 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:03:26.599434 13052 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:03:26.603636 13052 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:03:26.603709 13052 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:03:26.603730 13052 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:03:26.603749 13052 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:03:26.603765 13052 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:03:26.609310 13052 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:45343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786240941968412-1795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786240941968412-1795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.192.254:44101
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab
--principal=kudu/127.1.192.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.192.254:44101
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.192.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13052
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786240941968412-1795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:03:26.610477 13052 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:03:26.611521 13052 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:03:26.620358 13059 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:03:26.620358 13058 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:03:26.620380 13061 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:03:26.620687 13052 server_base.cc:1034] running on GCE node
I20260809 02:03:26.621403 13052 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:03:26.622653 13052 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:03:26.623848 13052 hybrid_clock.cc:648] HybridClock initialized: now 1786241006623814 us; error 46 us; skew 500 ppm
Aug 09 02:03:26 dist-test-slave-ss4n krb5kdc[13037](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241006, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:03:26.627046 13052 init.cc:377] Logged in from keytab as kudu/127.1.192.254@KRBTEST.COM (short username kudu)
I20260809 02:03:26.628226 13052 webserver.cc:460] Webserver started at http://127.1.192.254:35067/ using document root <none> and password file <none>
I20260809 02:03:26.629025 13052 fs_manager.cc:359] Metadata directory not provided
I20260809 02:03:26.629144 13052 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:03:26.629426 13052 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:03:26.631866 13052 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786240941968412-1795-0/minicluster-data/master-0/data/instance:
uuid: "1b0e139fc35b41ae9f8616780994f3c7"
format_stamp: "Formatted at 2026-08-09 02:03:26 on dist-test-slave-ss4n"
server_key: "57ab10aa90e6c280ec4848001ebe72c5"
server_key_iv: "ae95adc5b82fbb41ddd49c94e9299005"
server_key_version: "encryptionkey@0"
I20260809 02:03:26.632565 13052 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786240941968412-1795-0/minicluster-data/master-0/wal/instance:
uuid: "1b0e139fc35b41ae9f8616780994f3c7"
format_stamp: "Formatted at 2026-08-09 02:03:26 on dist-test-slave-ss4n"
server_key: "57ab10aa90e6c280ec4848001ebe72c5"
server_key_iv: "ae95adc5b82fbb41ddd49c94e9299005"
server_key_version: "encryptionkey@0"
I20260809 02:03:26.637094 13052 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260809 02:03:26.640038 13068 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:03:26.641355 13052 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 02:03:26.641551 13052 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786240941968412-1795-0/minicluster-data/master-0/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786240941968412-1795-0/minicluster-data/master-0/wal
uuid: "1b0e139fc35b41ae9f8616780994f3c7"
format_stamp: "Formatted at 2026-08-09 02:03:26 on dist-test-slave-ss4n"
server_key: "57ab10aa90e6c280ec4848001ebe72c5"
server_key_iv: "ae95adc5b82fbb41ddd49c94e9299005"
server_key_version: "encryptionkey@0"
I20260809 02:03:26.641712 13052 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786240941968412-1795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786240941968412-1795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786240941968412-1795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:03:26.661229 13052 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:03:26.664916 13052 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:03:26.665148 13052 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:03:26.672598 13052 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.254:44101
I20260809 02:03:26.672896 13122 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.254:44101 every 8 connection(s)
I20260809 02:03:26.673792 13052 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
I20260809 02:03:26.676544 13123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:03:26.682782 13123 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b0e139fc35b41ae9f8616780994f3c7: Bootstrap starting.
I20260809 02:03:26.683259 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 13052
I20260809 02:03:26.683362 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786240941968412-1795-0/minicluster-data/master-0/wal/instance
I20260809 02:03:26.683624 1795 external_mini_cluster.cc:1428] Setting key 7d813a80bacce8aac662622a349458ef
I20260809 02:03:26.685724 13123 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1b0e139fc35b41ae9f8616780994f3c7: Neither blocks nor log segments found. Creating new log.
I20260809 02:03:26.686534 13123 log.cc:824] T 00000000000000000000000000000000 P 1b0e139fc35b41ae9f8616780994f3c7: Log is configured to *not* fsync() on all Append() calls
I20260809 02:03:26.688601 13123 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b0e139fc35b41ae9f8616780994f3c7: No bootstrap required, opened a new log
Aug 09 02:03:26 dist-test-slave-ss4n krb5kdc[13037](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241006, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.192.254@KRBTEST.COM
I20260809 02:03:26.692998 13123 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1b0e139fc35b41ae9f8616780994f3c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b0e139fc35b41ae9f8616780994f3c7" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 44101 } }
I20260809 02:03:26.693324 13123 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1b0e139fc35b41ae9f8616780994f3c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:03:26.693403 13123 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1b0e139fc35b41ae9f8616780994f3c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b0e139fc35b41ae9f8616780994f3c7, State: Initialized, Role: FOLLOWER
I20260809 02:03:26.694010 13123 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1b0e139fc35b41ae9f8616780994f3c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b0e139fc35b41ae9f8616780994f3c7" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 44101 } }
I20260809 02:03:26.694211 13123 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1b0e139fc35b41ae9f8616780994f3c7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:03:26.694314 13123 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1b0e139fc35b41ae9f8616780994f3c7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:03:26.694433 13123 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1b0e139fc35b41ae9f8616780994f3c7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:03:26.695631 13123 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1b0e139fc35b41ae9f8616780994f3c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b0e139fc35b41ae9f8616780994f3c7" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 44101 } }
I20260809 02:03:26.696026 13123 leader_election.cc:302] T 00000000000000000000000000000000 P 1b0e139fc35b41ae9f8616780994f3c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b0e139fc35b41ae9f8616780994f3c7; no voters:
I20260809 02:03:26.696445 13123 leader_election.cc:288] T 00000000000000000000000000000000 P 1b0e139fc35b41ae9f8616780994f3c7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:03:26.696537 13128 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1b0e139fc35b41ae9f8616780994f3c7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:03:26.696789 13128 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1b0e139fc35b41ae9f8616780994f3c7 [term 1 LEADER]: Becoming Leader. State: Replica: 1b0e139fc35b41ae9f8616780994f3c7, State: Running, Role: LEADER
I20260809 02:03:26.697124 13128 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1b0e139fc35b41ae9f8616780994f3c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b0e139fc35b41ae9f8616780994f3c7" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 44101 } }
I20260809 02:03:26.698071 13123 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1b0e139fc35b41ae9f8616780994f3c7 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:03:26.699177 13129 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b0e139fc35b41ae9f8616780994f3c7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1b0e139fc35b41ae9f8616780994f3c7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b0e139fc35b41ae9f8616780994f3c7" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 44101 } } }
I20260809 02:03:26.699317 13129 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b0e139fc35b41ae9f8616780994f3c7 [sys.catalog]: This master's current role is: LEADER
I20260809 02:03:26.699548 13126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:26.685072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53188 (local address 127.1.192.254:44101)
0809 02:03:26.685605 (+ 533us) server_negotiation.cc:206] Beginning negotiation
0809 02:03:26.685621 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 02:03:26.685676 (+ 55us) server_negotiation.cc:407] Connection header received
0809 02:03:26.686377 (+ 701us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:03:26.686394 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:03:26.686772 (+ 378us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:03:26.687101 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:03:26.687970 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:26.689018 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:26.689930 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:26.690399 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:26.690585 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:26.693601 (+ 3016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:03:26.693606 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:03:26.693620 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:03:26.693654 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:03:26.695770 (+ 2116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:03:26.696256 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:03:26.696262 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:03:26.696269 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:03:26.696423 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:03:26.696854 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:03:26.696858 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:03:26.696860 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:03:26.697451 (+ 591us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:03:26.697707 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0809 02:03:26.698035 (+ 328us) server_negotiation.cc:299] Negotiation successful
0809 02:03:26.698378 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":163,"threads_started":1}
I20260809 02:03:26.699630 13130 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b0e139fc35b41ae9f8616780994f3c7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1b0e139fc35b41ae9f8616780994f3c7. Latest consensus state: current_term: 1 leader_uuid: "1b0e139fc35b41ae9f8616780994f3c7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b0e139fc35b41ae9f8616780994f3c7" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 44101 } } }
I20260809 02:03:26.699748 13130 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b0e139fc35b41ae9f8616780994f3c7 [sys.catalog]: This master's current role is: LEADER
I20260809 02:03:26.699996 13137 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:03:26.703295 13137 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:03:26.709223 13137 catalog_manager.cc:1347] Generated new cluster ID: fde8a1f5cf6b43dcaa26958fa1af7ff9
I20260809 02:03:26.709308 13137 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:03:26.727926 13137 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:03:26.729413 13137 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:03:26.742923 13137 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1b0e139fc35b41ae9f8616780994f3c7: Generated new TSK 0
I20260809 02:03:26.743745 13137 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.192.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.192.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.192.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.193.keytab.
Entry for principal kudu/127.1.192.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.193.keytab.
I20260809 02:03:26.781915 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786240941968412-1795-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786240941968412-1795-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786240941968412-1795-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786240941968412-1795-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.192.193:0
--local_ip_for_outbound_sockets=127.1.192.193
--webserver_interface=127.1.192.193
--webserver_port=0
--tserver_master_addrs=127.1.192.254:44101
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.193.keytab
--principal=kudu/127.1.192.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:45343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:03:26.926578 13150 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:03:26.926890 13150 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:03:26.926977 13150 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:03:26.931356 13150 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:03:26.931464 13150 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:03:26.931563 13150 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.192.193
I20260809 02:03:26.936990 13150 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:45343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786240941968412-1795-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786240941968412-1795-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.193.keytab
--principal=kudu/127.1.192.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.192.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786240941968412-1795-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.192.193
--webserver_port=0
--tserver_master_addrs=127.1.192.254:44101
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13150
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.192.193
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786240941968412-1795-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:03:26.938262 13150 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:03:26.939357 13150 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:03:26.949894 13158 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:03:26.949889 13156 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:03:26.950019 13155 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:03:26.950116 13150 server_base.cc:1034] running on GCE node
I20260809 02:03:26.950579 13150 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:03:26.951277 13150 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:03:26.952455 13150 hybrid_clock.cc:648] HybridClock initialized: now 1786241006952433 us; error 35 us; skew 500 ppm
Aug 09 02:03:26 dist-test-slave-ss4n krb5kdc[13037](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241006, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:03:26.956162 13150 init.cc:377] Logged in from keytab as kudu/127.1.192.193@KRBTEST.COM (short username kudu)
I20260809 02:03:26.957448 13150 webserver.cc:460] Webserver started at http://127.1.192.193:39437/ using document root <none> and password file <none>
I20260809 02:03:26.958130 13150 fs_manager.cc:359] Metadata directory not provided
I20260809 02:03:26.958201 13150 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:03:26.958458 13150 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:03:26.960783 13150 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786240941968412-1795-0/minicluster-data/ts-0/data/instance:
uuid: "ab2829755add43909af910c8aa0b60cf"
format_stamp: "Formatted at 2026-08-09 02:03:26 on dist-test-slave-ss4n"
server_key: "4b2232ff3e8dbcefec2ddc8f0c75fc34"
server_key_iv: "32ef1c04ee79f21f719fea9ea7d3fa90"
server_key_version: "encryptionkey@0"
I20260809 02:03:26.961424 13150 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786240941968412-1795-0/minicluster-data/ts-0/wal/instance:
uuid: "ab2829755add43909af910c8aa0b60cf"
format_stamp: "Formatted at 2026-08-09 02:03:26 on dist-test-slave-ss4n"
server_key: "4b2232ff3e8dbcefec2ddc8f0c75fc34"
server_key_iv: "32ef1c04ee79f21f719fea9ea7d3fa90"
server_key_version: "encryptionkey@0"
I20260809 02:03:26.965637 13150 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260809 02:03:26.968158 13165 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:03:26.969525 13150 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 02:03:26.969688 13150 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786240941968412-1795-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786240941968412-1795-0/minicluster-data/ts-0/wal
uuid: "ab2829755add43909af910c8aa0b60cf"
format_stamp: "Formatted at 2026-08-09 02:03:26 on dist-test-slave-ss4n"
server_key: "4b2232ff3e8dbcefec2ddc8f0c75fc34"
server_key_iv: "32ef1c04ee79f21f719fea9ea7d3fa90"
server_key_version: "encryptionkey@0"
I20260809 02:03:26.969805 13150 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786240941968412-1795-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786240941968412-1795-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786240941968412-1795-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:03:26.986923 13150 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:03:26.991775 13150 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:03:26.992213 13150 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:03:26.992986 13150 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:03:26.994100 13150 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:03:26.994191 13150 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:03:26.994249 13150 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:03:26.994282 13150 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:03:27.003883 13150 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.193:43971
I20260809 02:03:27.003907 13291 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.193:43971 every 8 connection(s)
I20260809 02:03:27.004805 13150 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786240941968412-1795-0/minicluster-data/ts-0/data/info.pb
I20260809 02:03:27.010951 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 13150
I20260809 02:03:27.011106 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786240941968412-1795-0/minicluster-data/ts-0/wal/instance
I20260809 02:03:27.011417 1795 external_mini_cluster.cc:1428] Setting key 610818d514a796c5c607f6a5265fd61e
Aug 09 02:03:27 dist-test-slave-ss4n krb5kdc[13037](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241006, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.193@KRBTEST.COM for kudu/127.1.192.254@KRBTEST.COM
I20260809 02:03:27.019800 13126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:27.006701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.193:35053 (local address 127.1.192.254:44101)
0809 02:03:27.006870 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0809 02:03:27.006875 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:03:27.007969 (+ 1094us) server_negotiation.cc:407] Connection header received
0809 02:03:27.008895 (+ 926us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:03:27.008898 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:03:27.008958 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:03:27.009064 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:03:27.010369 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:27.010907 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:27.012213 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:27.012628 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:27.012763 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:27.015419 (+ 2656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:03:27.015422 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:03:27.015424 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:03:27.015449 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:03:27.017172 (+ 1723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:03:27.017910 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:03:27.017913 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:03:27.017914 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:03:27.018011 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:03:27.018453 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:03:27.018456 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:03:27.018457 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:03:27.018706 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:03:27.018794 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 02:03:27.019495 (+ 701us) server_negotiation.cc:299] Negotiation successful
0809 02:03:27.019658 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":28,"server-negotiator.queue_time_us":65}
I20260809 02:03:27.021009 13295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:27.007157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:44101 (local address 127.1.192.193:35053)
0809 02:03:27.007806 (+ 649us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:27.007849 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 02:03:27.008685 (+ 836us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:03:27.009259 (+ 574us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:03:27.009265 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:03:27.009725 (+ 460us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:03:27.010215 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:27.010229 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:27.011045 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:27.011054 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:27.012076 (+ 1022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:27.012087 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:27.012821 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:27.012826 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:27.013003 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:27.013719 (+ 716us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:03:27.013736 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:03:27.015238 (+ 1502us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:03:27.017301 (+ 2063us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:03:27.017310 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:03:27.017328 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:03:27.017780 (+ 452us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:03:27.018115 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:03:27.018121 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:03:27.018126 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:03:27.018340 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:03:27.018829 (+ 489us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:03:27.018838 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:03:27.019210 (+ 372us) client_negotiation.cc:769] Sending connection context
0809 02:03:27.019444 (+ 234us) client_negotiation.cc:240] Negotiation successful
0809 02:03:27.019842 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":110,"threads_started":1}
I20260809 02:03:27.022549 13293 heartbeater.cc:347] Connected to a master server at 127.1.192.254:44101
I20260809 02:03:27.022831 13293 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.1.192.194@KRBTEST.COM; defaulting to no policy
I20260809 02:03:27.023373 13293 heartbeater.cc:511] Master 127.1.192.254:44101 requested a full tablet report, sending...
I20260809 02:03:27.025065 13085 ts_manager.cc:194] Registered new tserver with Master: ab2829755add43909af910c8aa0b60cf (127.1.192.193:43971)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.192.194@KRBTEST.COM" created.
I20260809 02:03:27.026721 13085 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.192.193@KRBTEST.COM'} at 127.1.192.193:35053
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.192.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.194.keytab.
Entry for principal kudu/127.1.192.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.194.keytab.
I20260809 02:03:27.045989 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786240941968412-1795-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786240941968412-1795-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786240941968412-1795-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786240941968412-1795-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.192.194:0
--local_ip_for_outbound_sockets=127.1.192.194
--webserver_interface=127.1.192.194
--webserver_port=0
--tserver_master_addrs=127.1.192.254:44101
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.194.keytab
--principal=kudu/127.1.192.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:45343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:03:27.158581 13299 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:03:27.158910 13299 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:03:27.158988 13299 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:03:27.163192 13299 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:03:27.163298 13299 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:03:27.163394 13299 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.192.194
I20260809 02:03:27.169749 13299 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:45343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786240941968412-1795-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786240941968412-1795-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.194.keytab
--principal=kudu/127.1.192.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.192.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786240941968412-1795-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.192.194
--webserver_port=0
--tserver_master_addrs=127.1.192.254:44101
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13299
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.192.194
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786240941968412-1795-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:03:27.170938 13299 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:03:27.172000 13299 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:03:27.182324 13305 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:03:27.182235 13304 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:03:27.182235 13308 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:03:27.182621 13299 server_base.cc:1034] running on GCE node
I20260809 02:03:27.183056 13299 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:03:27.183713 13299 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:03:27.184903 13299 hybrid_clock.cc:648] HybridClock initialized: now 1786241007184869 us; error 47 us; skew 500 ppm
Aug 09 02:03:27 dist-test-slave-ss4n krb5kdc[13037](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241007, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:03:27.187788 13299 init.cc:377] Logged in from keytab as kudu/127.1.192.194@KRBTEST.COM (short username kudu)
I20260809 02:03:27.188661 13299 webserver.cc:460] Webserver started at http://127.1.192.194:40369/ using document root <none> and password file <none>
I20260809 02:03:27.189186 13299 fs_manager.cc:359] Metadata directory not provided
I20260809 02:03:27.189245 13299 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:03:27.189409 13299 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:03:27.191470 13299 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786240941968412-1795-0/minicluster-data/ts-1/data/instance:
uuid: "b32ae86eec844d9f9a48e020324e4b6f"
format_stamp: "Formatted at 2026-08-09 02:03:27 on dist-test-slave-ss4n"
server_key: "83692fb7cbf19fecc935ddfbf668baf5"
server_key_iv: "118ab3bd42d0f7a1b8e2c257be06f8c2"
server_key_version: "encryptionkey@0"
I20260809 02:03:27.191982 13299 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786240941968412-1795-0/minicluster-data/ts-1/wal/instance:
uuid: "b32ae86eec844d9f9a48e020324e4b6f"
format_stamp: "Formatted at 2026-08-09 02:03:27 on dist-test-slave-ss4n"
server_key: "83692fb7cbf19fecc935ddfbf668baf5"
server_key_iv: "118ab3bd42d0f7a1b8e2c257be06f8c2"
server_key_version: "encryptionkey@0"
I20260809 02:03:27.195737 13299 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 02:03:27.198838 13318 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:03:27.200444 13299 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 02:03:27.200546 13299 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786240941968412-1795-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786240941968412-1795-0/minicluster-data/ts-1/wal
uuid: "b32ae86eec844d9f9a48e020324e4b6f"
format_stamp: "Formatted at 2026-08-09 02:03:27 on dist-test-slave-ss4n"
server_key: "83692fb7cbf19fecc935ddfbf668baf5"
server_key_iv: "118ab3bd42d0f7a1b8e2c257be06f8c2"
server_key_version: "encryptionkey@0"
I20260809 02:03:27.200634 13299 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786240941968412-1795-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786240941968412-1795-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786240941968412-1795-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:03:27.221323 13299 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:03:27.224902 13299 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:03:27.225072 13299 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:03:27.225713 13299 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:03:27.226533 13299 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:03:27.226584 13299 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:03:27.226625 13299 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:03:27.226641 13299 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:03:27.236025 13299 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.194:33729
I20260809 02:03:27.236058 13442 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.194:33729 every 8 connection(s)
I20260809 02:03:27.236966 13299 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786240941968412-1795-0/minicluster-data/ts-1/data/info.pb
I20260809 02:03:27.243788 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 13299
I20260809 02:03:27.243883 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786240941968412-1795-0/minicluster-data/ts-1/wal/instance
I20260809 02:03:27.244143 1795 external_mini_cluster.cc:1428] Setting key a943059de1dbb5c6e31ff7d1dc4290df
Aug 09 02:03:27 dist-test-slave-ss4n krb5kdc[13037](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241007, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.194@KRBTEST.COM for kudu/127.1.192.254@KRBTEST.COM
I20260809 02:03:27.251749 13126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:27.238832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.194:44567 (local address 127.1.192.254:44101)
0809 02:03:27.238983 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0809 02:03:27.238987 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:03:27.239818 (+ 831us) server_negotiation.cc:407] Connection header received
0809 02:03:27.241161 (+ 1343us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:03:27.241163 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:03:27.241213 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:03:27.241291 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:03:27.242580 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:27.243224 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:27.244504 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:27.244867 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:27.245012 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:27.247982 (+ 2970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:03:27.247986 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:03:27.247989 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:03:27.248020 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:03:27.249618 (+ 1598us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:03:27.250149 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:03:27.250153 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:03:27.250155 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:03:27.250243 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:03:27.250631 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:03:27.250636 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:03:27.250637 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:03:27.250837 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:03:27.250947 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 02:03:27.251453 (+ 506us) server_negotiation.cc:299] Negotiation successful
0809 02:03:27.251584 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 02:03:27.252627 13445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:27.239185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:44101 (local address 127.1.192.194:44567)
0809 02:03:27.239644 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:27.239691 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 02:03:27.240956 (+ 1265us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:03:27.241454 (+ 498us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:03:27.241461 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:03:27.241898 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:03:27.242410 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:27.242423 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:27.243347 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:27.243350 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:27.244372 (+ 1022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:27.244382 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:27.245059 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:27.245063 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:27.245253 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:27.245932 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:03:27.245958 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:03:27.247804 (+ 1846us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:03:27.249740 (+ 1936us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:03:27.249746 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:03:27.249763 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:03:27.250034 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:03:27.250338 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:03:27.250342 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:03:27.250344 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:03:27.250518 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:03:27.250946 (+ 428us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:03:27.250953 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:03:27.251212 (+ 259us) client_negotiation.cc:769] Sending connection context
0809 02:03:27.251408 (+ 196us) client_negotiation.cc:240] Negotiation successful
0809 02:03:27.251764 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":117,"threads_started":1}
I20260809 02:03:27.253913 13443 heartbeater.cc:347] Connected to a master server at 127.1.192.254:44101
I20260809 02:03:27.254185 13443 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.1.192.195@KRBTEST.COM; defaulting to no policy
I20260809 02:03:27.254724 13443 heartbeater.cc:511] Master 127.1.192.254:44101 requested a full tablet report, sending...
I20260809 02:03:27.255820 13085 ts_manager.cc:194] Registered new tserver with Master: b32ae86eec844d9f9a48e020324e4b6f (127.1.192.194:33729)
I20260809 02:03:27.256403 13085 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.192.194@KRBTEST.COM'} at 127.1.192.194:44567
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.192.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.192.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.195.keytab.
Entry for principal kudu/127.1.192.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.195.keytab.
I20260809 02:03:27.276903 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786240941968412-1795-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786240941968412-1795-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786240941968412-1795-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786240941968412-1795-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.192.195:0
--local_ip_for_outbound_sockets=127.1.192.195
--webserver_interface=127.1.192.195
--webserver_port=0
--tserver_master_addrs=127.1.192.254:44101
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.195.keytab
--principal=kudu/127.1.192.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:45343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:03:27.393771 13448 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:03:27.394043 13448 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:03:27.394090 13448 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:03:27.398181 13448 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:03:27.398249 13448 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:03:27.398308 13448 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.192.195
I20260809 02:03:27.404016 13448 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:45343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786240941968412-1795-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786240941968412-1795-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.195.keytab
--principal=kudu/127.1.192.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.192.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786240941968412-1795-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.192.195
--webserver_port=0
--tserver_master_addrs=127.1.192.254:44101
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13448
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.192.195
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786240941968412-1795-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:03:27.405153 13448 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:03:27.406252 13448 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:03:27.416268 13453 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:03:27.416275 13454 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:03:27.416275 13456 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:03:27.416644 13448 server_base.cc:1034] running on GCE node
I20260809 02:03:27.417135 13448 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:03:27.417994 13448 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:03:27.419221 13448 hybrid_clock.cc:648] HybridClock initialized: now 1786241007419213 us; error 33 us; skew 500 ppm
Aug 09 02:03:27 dist-test-slave-ss4n krb5kdc[13037](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241007, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:03:27.422501 13448 init.cc:377] Logged in from keytab as kudu/127.1.192.195@KRBTEST.COM (short username kudu)
I20260809 02:03:27.423413 13448 webserver.cc:460] Webserver started at http://127.1.192.195:45489/ using document root <none> and password file <none>
I20260809 02:03:27.424055 13448 fs_manager.cc:359] Metadata directory not provided
I20260809 02:03:27.424126 13448 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:03:27.424350 13448 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:03:27.426497 13448 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786240941968412-1795-0/minicluster-data/ts-2/data/instance:
uuid: "99339ad25be24f1ebc8448c52077feb7"
format_stamp: "Formatted at 2026-08-09 02:03:27 on dist-test-slave-ss4n"
server_key: "9995a443b0c7cd624c88c1c5c9a2c607"
server_key_iv: "198c9282cb5ffdb20fff8a1e5a03d9ec"
server_key_version: "encryptionkey@0"
I20260809 02:03:27.427089 13448 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786240941968412-1795-0/minicluster-data/ts-2/wal/instance:
uuid: "99339ad25be24f1ebc8448c52077feb7"
format_stamp: "Formatted at 2026-08-09 02:03:27 on dist-test-slave-ss4n"
server_key: "9995a443b0c7cd624c88c1c5c9a2c607"
server_key_iv: "198c9282cb5ffdb20fff8a1e5a03d9ec"
server_key_version: "encryptionkey@0"
I20260809 02:03:27.431095 13448 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 02:03:27.433733 13464 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:03:27.435204 13448 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 02:03:27.435364 13448 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786240941968412-1795-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786240941968412-1795-0/minicluster-data/ts-2/wal
uuid: "99339ad25be24f1ebc8448c52077feb7"
format_stamp: "Formatted at 2026-08-09 02:03:27 on dist-test-slave-ss4n"
server_key: "9995a443b0c7cd624c88c1c5c9a2c607"
server_key_iv: "198c9282cb5ffdb20fff8a1e5a03d9ec"
server_key_version: "encryptionkey@0"
I20260809 02:03:27.435564 13448 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786240941968412-1795-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786240941968412-1795-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786240941968412-1795-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:03:27.447365 13448 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:03:27.451148 13448 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:03:27.451383 13448 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:03:27.452034 13448 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:03:27.452943 13448 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:03:27.453023 13448 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:03:27.453102 13448 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:03:27.453161 13448 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:03:27.463119 13448 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.195:41611
I20260809 02:03:27.463143 13587 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.195:41611 every 8 connection(s)
I20260809 02:03:27.464175 13448 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786240941968412-1795-0/minicluster-data/ts-2/data/info.pb
I20260809 02:03:27.464938 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 13448
I20260809 02:03:27.465091 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786240941968412-1795-0/minicluster-data/ts-2/wal/instance
I20260809 02:03:27.465423 1795 external_mini_cluster.cc:1428] Setting key b3bf8e699aede74866a2ebefe388ec2d
Aug 09 02:03:27 dist-test-slave-ss4n krb5kdc[13037](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241007, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.195@KRBTEST.COM for kudu/127.1.192.254@KRBTEST.COM
I20260809 02:03:27.479084 13126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:27.466300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.195:43769 (local address 127.1.192.254:44101)
0809 02:03:27.466450 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0809 02:03:27.466455 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:03:27.467427 (+ 972us) server_negotiation.cc:407] Connection header received
0809 02:03:27.468921 (+ 1494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:03:27.468924 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:03:27.468971 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:03:27.469077 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:03:27.470311 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:27.470862 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:27.471968 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:27.472203 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:27.472287 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:27.475192 (+ 2905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:03:27.475195 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:03:27.475197 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:03:27.475224 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:03:27.476815 (+ 1591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:03:27.477354 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:03:27.477356 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:03:27.477358 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:03:27.477439 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:03:27.477862 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:03:27.477864 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:03:27.477866 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:03:27.478097 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:03:27.478227 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0809 02:03:27.478775 (+ 548us) server_negotiation.cc:299] Negotiation successful
0809 02:03:27.478907 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 02:03:27.480156 13590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:27.466768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:44101 (local address 127.1.192.195:43769)
0809 02:03:27.467266 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:27.467321 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0809 02:03:27.468695 (+ 1374us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:03:27.469227 (+ 532us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:03:27.469234 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:03:27.469690 (+ 456us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:03:27.470180 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:27.470194 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:27.470974 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:27.470976 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:27.471848 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:27.471857 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:27.472340 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:27.472343 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:27.472485 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:27.473178 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:03:27.473203 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:03:27.475006 (+ 1803us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:03:27.476943 (+ 1937us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:03:27.476948 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:03:27.476962 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:03:27.477256 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:03:27.477607 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:03:27.477615 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:03:27.477617 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:03:27.477772 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:03:27.478220 (+ 448us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:03:27.478228 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:03:27.478500 (+ 272us) client_negotiation.cc:769] Sending connection context
0809 02:03:27.478729 (+ 229us) client_negotiation.cc:240] Negotiation successful
0809 02:03:27.479088 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":139,"threads_started":1}
I20260809 02:03:27.481266 13588 heartbeater.cc:347] Connected to a master server at 127.1.192.254:44101
I20260809 02:03:27.481626 13588 heartbeater.cc:464] Registering TS with master...
I20260809 02:03:27.482164 13588 heartbeater.cc:511] Master 127.1.192.254:44101 requested a full tablet report, sending...
I20260809 02:03:27.483330 13085 ts_manager.cc:194] Registered new tserver with Master: 99339ad25be24f1ebc8448c52077feb7 (127.1.192.195:41611)
I20260809 02:03:27.483914 13085 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.192.195@KRBTEST.COM'} at 127.1.192.195:43769
I20260809 02:03:27.492239 1795 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 02:03:27.502334 13126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:27.494370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53196 (local address 127.1.192.254:44101)
0809 02:03:27.494622 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0809 02:03:27.494628 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:03:27.494768 (+ 140us) server_negotiation.cc:407] Connection header received
0809 02:03:27.494866 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:03:27.494868 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:03:27.494912 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:03:27.494994 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:03:27.495637 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:27.496174 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:27.497085 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:27.497304 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:27.497385 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:27.499040 (+ 1655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:03:27.499044 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:03:27.499046 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:03:27.499111 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:03:27.500661 (+ 1550us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:03:27.501110 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:03:27.501112 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:03:27.501113 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:03:27.501193 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:03:27.501452 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:03:27.501455 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:03:27.501456 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:03:27.501755 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:03:27.501866 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 02:03:27.502102 (+ 236us) server_negotiation.cc:299] Negotiation successful
0809 02:03:27.502212 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160}
I20260809 02:03:27.505329 13085 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53196:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 02:03:27.508472 13085 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:03:27.523118 13599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:27.517089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.193:43971 (local address 127.0.0.1:46910)
0809 02:03:27.518460 (+ 1371us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:27.518473 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:03:27.518561 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:03:27.519050 (+ 489us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:03:27.519053 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:03:27.519070 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:03:27.519180 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:27.519187 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:27.520324 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:27.520330 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:27.521667 (+ 1337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:27.521677 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:27.522523 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:27.522528 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:27.522641 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:27.522684 (+ 43us) client_negotiation.cc:769] Sending connection context
0809 02:03:27.522823 (+ 139us) client_negotiation.cc:240] Negotiation successful
0809 02:03:27.522954 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1318,"mutex_wait_us":760,"thread_start_us":99,"threads_started":1}
I20260809 02:03:27.523128 13604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:27.517833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46910 (local address 127.1.192.193:43971)
0809 02:03:27.518324 (+ 491us) server_negotiation.cc:206] Beginning negotiation
0809 02:03:27.518343 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0809 02:03:27.518558 (+ 215us) server_negotiation.cc:407] Connection header received
0809 02:03:27.518665 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:03:27.518670 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:03:27.518835 (+ 165us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:03:27.518964 (+ 129us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:03:27.519320 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:27.520184 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:27.521794 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:27.522374 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:27.522517 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:27.522680 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0809 02:03:27.522865 (+ 185us) server_negotiation.cc:299] Negotiation successful
0809 02:03:27.522959 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":387,"thread_start_us":68,"threads_started":1}
I20260809 02:03:27.523934 13600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:27.517089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.195:41611 (local address 127.0.0.1:46082)
0809 02:03:27.518068 (+ 979us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:27.518122 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 02:03:27.518265 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:03:27.518883 (+ 618us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:03:27.518889 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:03:27.518912 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:03:27.519058 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:27.519068 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:27.520404 (+ 1336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:27.520407 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:27.521773 (+ 1366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:27.521786 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:27.523066 (+ 1280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:27.523070 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:27.523160 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:27.523189 (+ 29us) client_negotiation.cc:769] Sending connection context
0809 02:03:27.523704 (+ 515us) client_negotiation.cc:240] Negotiation successful
0809 02:03:27.523783 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":805,"mutex_wait_us":24,"thread_start_us":109,"threads_started":1}
I20260809 02:03:27.524012 13602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:27.517567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43432 (local address 127.1.192.194:33729)
0809 02:03:27.518107 (+ 540us) server_negotiation.cc:206] Beginning negotiation
0809 02:03:27.518119 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 02:03:27.518434 (+ 315us) server_negotiation.cc:407] Connection header received
0809 02:03:27.518897 (+ 463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:03:27.518903 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:03:27.519084 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:03:27.519233 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:03:27.519697 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:27.520760 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:27.522473 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:27.523352 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:27.523463 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:27.523602 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0809 02:03:27.523785 (+ 183us) server_negotiation.cc:299] Negotiation successful
0809 02:03:27.523871 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":458,"thread_start_us":131,"threads_started":1}
I20260809 02:03:27.524156 13601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:27.517367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46082 (local address 127.1.192.195:41611)
0809 02:03:27.517936 (+ 569us) server_negotiation.cc:206] Beginning negotiation
0809 02:03:27.517949 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 02:03:27.518398 (+ 449us) server_negotiation.cc:407] Connection header received
0809 02:03:27.518487 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:03:27.518492 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:03:27.518703 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:03:27.518857 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:03:27.519169 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:27.520247 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:27.521919 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:27.522785 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:27.522881 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:27.523007 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0809 02:03:27.523871 (+ 864us) server_negotiation.cc:299] Negotiation successful
0809 02:03:27.523978 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":453,"thread_start_us":131,"threads_started":1}
I20260809 02:03:27.524444 13603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:27.517383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.194:33729 (local address 127.0.0.1:43432)
0809 02:03:27.518337 (+ 954us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:27.518352 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 02:03:27.518738 (+ 386us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:03:27.519436 (+ 698us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:03:27.519438 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:03:27.519457 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:03:27.519554 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:27.519561 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:27.520936 (+ 1375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:27.520939 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:27.522331 (+ 1392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:27.522341 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:27.523534 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:27.523537 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:27.523642 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:27.523675 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 02:03:27.523743 (+ 68us) client_negotiation.cc:240] Negotiation successful
0809 02:03:27.523791 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":717,"mutex_wait_us":4,"thread_start_us":128,"threads_started":1}
I20260809 02:03:27.526648 13516 tablet_service.cc:1467] Processing CreateTablet for tablet 9be3f52aeb204599bc33d8f26d72b11d (DEFAULT_TABLE table=test-table [id=09d657a73ae342f581065642ccdea692]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:03:27.526688 13220 tablet_service.cc:1467] Processing CreateTablet for tablet 9be3f52aeb204599bc33d8f26d72b11d (DEFAULT_TABLE table=test-table [id=09d657a73ae342f581065642ccdea692]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:03:27.526880 13368 tablet_service.cc:1467] Processing CreateTablet for tablet 9be3f52aeb204599bc33d8f26d72b11d (DEFAULT_TABLE table=test-table [id=09d657a73ae342f581065642ccdea692]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:03:27.527673 13516 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9be3f52aeb204599bc33d8f26d72b11d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:03:27.527729 13220 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9be3f52aeb204599bc33d8f26d72b11d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:03:27.527729 13368 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9be3f52aeb204599bc33d8f26d72b11d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:03:27.534411 13605 tablet_bootstrap.cc:492] T 9be3f52aeb204599bc33d8f26d72b11d P 99339ad25be24f1ebc8448c52077feb7: Bootstrap starting.
I20260809 02:03:27.534905 13607 tablet_bootstrap.cc:492] T 9be3f52aeb204599bc33d8f26d72b11d P b32ae86eec844d9f9a48e020324e4b6f: Bootstrap starting.
I20260809 02:03:27.537815 13607 tablet_bootstrap.cc:654] T 9be3f52aeb204599bc33d8f26d72b11d P b32ae86eec844d9f9a48e020324e4b6f: Neither blocks nor log segments found. Creating new log.
I20260809 02:03:27.537817 13606 tablet_bootstrap.cc:492] T 9be3f52aeb204599bc33d8f26d72b11d P ab2829755add43909af910c8aa0b60cf: Bootstrap starting.
I20260809 02:03:27.538758 13607 log.cc:824] T 9be3f52aeb204599bc33d8f26d72b11d P b32ae86eec844d9f9a48e020324e4b6f: Log is configured to *not* fsync() on all Append() calls
I20260809 02:03:27.538911 13605 tablet_bootstrap.cc:654] T 9be3f52aeb204599bc33d8f26d72b11d P 99339ad25be24f1ebc8448c52077feb7: Neither blocks nor log segments found. Creating new log.
I20260809 02:03:27.540077 13606 tablet_bootstrap.cc:654] T 9be3f52aeb204599bc33d8f26d72b11d P ab2829755add43909af910c8aa0b60cf: Neither blocks nor log segments found. Creating new log.
I20260809 02:03:27.540196 13605 log.cc:824] T 9be3f52aeb204599bc33d8f26d72b11d P 99339ad25be24f1ebc8448c52077feb7: Log is configured to *not* fsync() on all Append() calls
I20260809 02:03:27.540871 13606 log.cc:824] T 9be3f52aeb204599bc33d8f26d72b11d P ab2829755add43909af910c8aa0b60cf: Log is configured to *not* fsync() on all Append() calls
I20260809 02:03:27.541118 13607 tablet_bootstrap.cc:492] T 9be3f52aeb204599bc33d8f26d72b11d P b32ae86eec844d9f9a48e020324e4b6f: No bootstrap required, opened a new log
I20260809 02:03:27.541339 13607 ts_tablet_manager.cc:1397] T 9be3f52aeb204599bc33d8f26d72b11d P b32ae86eec844d9f9a48e020324e4b6f: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.005s
I20260809 02:03:27.542405 13605 tablet_bootstrap.cc:492] T 9be3f52aeb204599bc33d8f26d72b11d P 99339ad25be24f1ebc8448c52077feb7: No bootstrap required, opened a new log
I20260809 02:03:27.542595 13605 ts_tablet_manager.cc:1397] T 9be3f52aeb204599bc33d8f26d72b11d P 99339ad25be24f1ebc8448c52077feb7: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260809 02:03:27.542695 13606 tablet_bootstrap.cc:492] T 9be3f52aeb204599bc33d8f26d72b11d P ab2829755add43909af910c8aa0b60cf: No bootstrap required, opened a new log
I20260809 02:03:27.542878 13606 ts_tablet_manager.cc:1397] T 9be3f52aeb204599bc33d8f26d72b11d P ab2829755add43909af910c8aa0b60cf: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260809 02:03:27.545428 13607 raft_consensus.cc:348] T 9be3f52aeb204599bc33d8f26d72b11d P b32ae86eec844d9f9a48e020324e4b6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99339ad25be24f1ebc8448c52077feb7" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 41611 } } peers { permanent_uuid: "ab2829755add43909af910c8aa0b60cf" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 43971 } } peers { permanent_uuid: "b32ae86eec844d9f9a48e020324e4b6f" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 33729 } }
I20260809 02:03:27.545807 13607 raft_consensus.cc:374] T 9be3f52aeb204599bc33d8f26d72b11d P b32ae86eec844d9f9a48e020324e4b6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:03:27.545877 13607 raft_consensus.cc:729] T 9be3f52aeb204599bc33d8f26d72b11d P b32ae86eec844d9f9a48e020324e4b6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b32ae86eec844d9f9a48e020324e4b6f, State: Initialized, Role: FOLLOWER
I20260809 02:03:27.546380 13607 consensus_queue.cc:260] T 9be3f52aeb204599bc33d8f26d72b11d P b32ae86eec844d9f9a48e020324e4b6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99339ad25be24f1ebc8448c52077feb7" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 41611 } } peers { permanent_uuid: "ab2829755add43909af910c8aa0b60cf" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 43971 } } peers { permanent_uuid: "b32ae86eec844d9f9a48e020324e4b6f" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 33729 } }
I20260809 02:03:27.546587 13605 raft_consensus.cc:348] T 9be3f52aeb204599bc33d8f26d72b11d P 99339ad25be24f1ebc8448c52077feb7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99339ad25be24f1ebc8448c52077feb7" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 41611 } } peers { permanent_uuid: "ab2829755add43909af910c8aa0b60cf" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 43971 } } peers { permanent_uuid: "b32ae86eec844d9f9a48e020324e4b6f" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 33729 } }
I20260809 02:03:27.546857 13605 raft_consensus.cc:374] T 9be3f52aeb204599bc33d8f26d72b11d P 99339ad25be24f1ebc8448c52077feb7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:03:27.546916 13605 raft_consensus.cc:729] T 9be3f52aeb204599bc33d8f26d72b11d P 99339ad25be24f1ebc8448c52077feb7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99339ad25be24f1ebc8448c52077feb7, State: Initialized, Role: FOLLOWER
I20260809 02:03:27.547022 13606 raft_consensus.cc:348] T 9be3f52aeb204599bc33d8f26d72b11d P ab2829755add43909af910c8aa0b60cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99339ad25be24f1ebc8448c52077feb7" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 41611 } } peers { permanent_uuid: "ab2829755add43909af910c8aa0b60cf" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 43971 } } peers { permanent_uuid: "b32ae86eec844d9f9a48e020324e4b6f" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 33729 } }
I20260809 02:03:27.547297 13606 raft_consensus.cc:374] T 9be3f52aeb204599bc33d8f26d72b11d P ab2829755add43909af910c8aa0b60cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:03:27.547370 13606 raft_consensus.cc:729] T 9be3f52aeb204599bc33d8f26d72b11d P ab2829755add43909af910c8aa0b60cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab2829755add43909af910c8aa0b60cf, State: Initialized, Role: FOLLOWER
I20260809 02:03:27.547309 13605 consensus_queue.cc:260] T 9be3f52aeb204599bc33d8f26d72b11d P 99339ad25be24f1ebc8448c52077feb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99339ad25be24f1ebc8448c52077feb7" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 41611 } } peers { permanent_uuid: "ab2829755add43909af910c8aa0b60cf" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 43971 } } peers { permanent_uuid: "b32ae86eec844d9f9a48e020324e4b6f" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 33729 } }
I20260809 02:03:27.547470 13607 ts_tablet_manager.cc:1428] T 9be3f52aeb204599bc33d8f26d72b11d P b32ae86eec844d9f9a48e020324e4b6f: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260809 02:03:27.547768 13606 consensus_queue.cc:260] T 9be3f52aeb204599bc33d8f26d72b11d P ab2829755add43909af910c8aa0b60cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99339ad25be24f1ebc8448c52077feb7" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 41611 } } peers { permanent_uuid: "ab2829755add43909af910c8aa0b60cf" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 43971 } } peers { permanent_uuid: "b32ae86eec844d9f9a48e020324e4b6f" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 33729 } }
I20260809 02:03:27.547909 13443 heartbeater.cc:503] Master 127.1.192.254:44101 was elected leader, sending a full tablet report...
I20260809 02:03:27.547966 13588 heartbeater.cc:503] Master 127.1.192.254:44101 was elected leader, sending a full tablet report...
I20260809 02:03:27.548234 13605 ts_tablet_manager.cc:1428] T 9be3f52aeb204599bc33d8f26d72b11d P 99339ad25be24f1ebc8448c52077feb7: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260809 02:03:27.548676 13606 ts_tablet_manager.cc:1428] T 9be3f52aeb204599bc33d8f26d72b11d P ab2829755add43909af910c8aa0b60cf: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260809 02:03:27.549001 13293 heartbeater.cc:503] Master 127.1.192.254:44101 was elected leader, sending a full tablet report...
W20260809 02:03:27.715384 13589 tablet.cc:2369] T 9be3f52aeb204599bc33d8f26d72b11d P 99339ad25be24f1ebc8448c52077feb7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 02:03:27.738214 13444 tablet.cc:2369] T 9be3f52aeb204599bc33d8f26d72b11d P b32ae86eec844d9f9a48e020324e4b6f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 02:03:27.756233 13294 tablet.cc:2369] T 9be3f52aeb204599bc33d8f26d72b11d P ab2829755add43909af910c8aa0b60cf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 02:03:27.847466 13612 raft_consensus.cc:482] T 9be3f52aeb204599bc33d8f26d72b11d P b32ae86eec844d9f9a48e020324e4b6f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:03:27.847767 13612 raft_consensus.cc:504] T 9be3f52aeb204599bc33d8f26d72b11d P b32ae86eec844d9f9a48e020324e4b6f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99339ad25be24f1ebc8448c52077feb7" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 41611 } } peers { permanent_uuid: "ab2829755add43909af910c8aa0b60cf" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 43971 } } peers { permanent_uuid: "b32ae86eec844d9f9a48e020324e4b6f" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 33729 } }
I20260809 02:03:27.848948 13612 leader_election.cc:288] T 9be3f52aeb204599bc33d8f26d72b11d P b32ae86eec844d9f9a48e020324e4b6f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99339ad25be24f1ebc8448c52077feb7 (127.1.192.195:41611), ab2829755add43909af910c8aa0b60cf (127.1.192.193:43971)
I20260809 02:03:27.853117 13616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:27.849302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.195:41611 (local address 127.1.192.194:52249)
0809 02:03:27.849770 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:27.849782 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:03:27.849917 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:03:27.850201 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:03:27.850204 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:03:27.850220 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:03:27.850379 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:27.850385 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:27.851126 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:27.851130 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:27.851966 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:27.851975 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:27.852653 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:27.852658 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:27.852757 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:27.852788 (+ 31us) client_negotiation.cc:769] Sending connection context
0809 02:03:27.852880 (+ 92us) client_negotiation.cc:240] Negotiation successful
0809 02:03:27.852954 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":401,"mutex_wait_us":44,"thread_start_us":162,"threads_started":1}
I20260809 02:03:27.853097 13601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:27.849451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.194:52249 (local address 127.1.192.195:41611)
0809 02:03:27.849679 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0809 02:03:27.849684 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:03:27.849877 (+ 193us) server_negotiation.cc:407] Connection header received
0809 02:03:27.850043 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:03:27.850046 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:03:27.850095 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:03:27.850189 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:03:27.850498 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:27.851009 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:27.852102 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:27.852513 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:27.852653 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:27.852711 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 02:03:27.852898 (+ 187us) server_negotiation.cc:299] Negotiation successful
0809 02:03:27.852970 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135}
I20260809 02:03:27.853782 13604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:27.849417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.194:41131 (local address 127.1.192.193:43971)
0809 02:03:27.849585 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0809 02:03:27.849592 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 02:03:27.849845 (+ 253us) server_negotiation.cc:407] Connection header received
0809 02:03:27.850051 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:03:27.850055 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:03:27.850114 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:03:27.850210 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:03:27.850627 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:27.851192 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:27.852432 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:27.852909 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:27.853068 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:27.853140 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 02:03:27.853553 (+ 413us) server_negotiation.cc:299] Negotiation successful
0809 02:03:27.853638 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260809 02:03:27.853808 13615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:27.849286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.193:43971 (local address 127.1.192.194:41131)
0809 02:03:27.849717 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:27.849733 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 02:03:27.849908 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:03:27.850243 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:03:27.850246 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:03:27.850265 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:03:27.850485 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:27.850493 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:27.851328 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:27.851334 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:27.852271 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:03:27.852281 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:03:27.853056 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:03:27.853061 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:03:27.853378 (+ 317us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:27.853412 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 02:03:27.853537 (+ 125us) client_negotiation.cc:240] Negotiation successful
0809 02:03:27.853613 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":332,"mutex_wait_us":29,"thread_start_us":114,"threads_started":1}
I20260809 02:03:27.854074 13538 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9be3f52aeb204599bc33d8f26d72b11d" candidate_uuid: "b32ae86eec844d9f9a48e020324e4b6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99339ad25be24f1ebc8448c52077feb7" is_pre_election: true
I20260809 02:03:27.854511 13538 raft_consensus.cc:2454] T 9be3f52aeb204599bc33d8f26d72b11d P 99339ad25be24f1ebc8448c52077feb7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b32ae86eec844d9f9a48e020324e4b6f in term 0.
I20260809 02:03:27.854609 13242 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9be3f52aeb204599bc33d8f26d72b11d" candidate_uuid: "b32ae86eec844d9f9a48e020324e4b6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab2829755add43909af910c8aa0b60cf" is_pre_election: true
I20260809 02:03:27.854945 13242 raft_consensus.cc:2454] T 9be3f52aeb204599bc33d8f26d72b11d P ab2829755add43909af910c8aa0b60cf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b32ae86eec844d9f9a48e020324e4b6f in term 0.
I20260809 02:03:27.855075 13319 leader_election.cc:302] T 9be3f52aeb204599bc33d8f26d72b11d P b32ae86eec844d9f9a48e020324e4b6f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 99339ad25be24f1ebc8448c52077feb7, b32ae86eec844d9f9a48e020324e4b6f; no voters:
I20260809 02:03:27.855422 13612 raft_consensus.cc:2781] T 9be3f52aeb204599bc33d8f26d72b11d P b32ae86eec844d9f9a48e020324e4b6f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:03:27.855515 13612 raft_consensus.cc:482] T 9be3f52aeb204599bc33d8f26d72b11d P b32ae86eec844d9f9a48e020324e4b6f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:03:27.855547 13612 raft_consensus.cc:3037] T 9be3f52aeb204599bc33d8f26d72b11d P b32ae86eec844d9f9a48e020324e4b6f [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:03:27.856652 13612 raft_consensus.cc:504] T 9be3f52aeb204599bc33d8f26d72b11d P b32ae86eec844d9f9a48e020324e4b6f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99339ad25be24f1ebc8448c52077feb7" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 41611 } } peers { permanent_uuid: "ab2829755add43909af910c8aa0b60cf" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 43971 } } peers { permanent_uuid: "b32ae86eec844d9f9a48e020324e4b6f" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 33729 } }
I20260809 02:03:27.857059 13612 leader_election.cc:288] T 9be3f52aeb204599bc33d8f26d72b11d P b32ae86eec844d9f9a48e020324e4b6f [CANDIDATE]: Term 1 election: Requested vote from peers 99339ad25be24f1ebc8448c52077feb7 (127.1.192.195:41611), ab2829755add43909af910c8aa0b60cf (127.1.192.193:43971)
I20260809 02:03:27.857520 13538 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9be3f52aeb204599bc33d8f26d72b11d" candidate_uuid: "b32ae86eec844d9f9a48e020324e4b6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99339ad25be24f1ebc8448c52077feb7"
I20260809 02:03:27.857650 13242 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9be3f52aeb204599bc33d8f26d72b11d" candidate_uuid: "b32ae86eec844d9f9a48e020324e4b6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab2829755add43909af910c8aa0b60cf"
I20260809 02:03:27.857656 13538 raft_consensus.cc:3037] T 9be3f52aeb204599bc33d8f26d72b11d P 99339ad25be24f1ebc8448c52077feb7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:03:27.857826 13242 raft_consensus.cc:3037] T 9be3f52aeb204599bc33d8f26d72b11d P ab2829755add43909af910c8aa0b60cf [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:03:27.858820 13538 raft_consensus.cc:2454] T 9be3f52aeb204599bc33d8f26d72b11d P 99339ad25be24f1ebc8448c52077feb7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b32ae86eec844d9f9a48e020324e4b6f in term 1.
I20260809 02:03:27.859066 13242 raft_consensus.cc:2454] T 9be3f52aeb204599bc33d8f26d72b11d P ab2829755add43909af910c8aa0b60cf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b32ae86eec844d9f9a48e020324e4b6f in term 1.
I20260809 02:03:27.859100 13319 leader_election.cc:302] T 9be3f52aeb204599bc33d8f26d72b11d P b32ae86eec844d9f9a48e020324e4b6f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 99339ad25be24f1ebc8448c52077feb7, b32ae86eec844d9f9a48e020324e4b6f; no voters:
I20260809 02:03:27.859305 13612 raft_consensus.cc:2781] T 9be3f52aeb204599bc33d8f26d72b11d P b32ae86eec844d9f9a48e020324e4b6f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:03:27.859603 13612 raft_consensus.cc:686] T 9be3f52aeb204599bc33d8f26d72b11d P b32ae86eec844d9f9a48e020324e4b6f [term 1 LEADER]: Becoming Leader. State: Replica: b32ae86eec844d9f9a48e020324e4b6f, State: Running, Role: LEADER
I20260809 02:03:27.859998 13612 consensus_queue.cc:237] T 9be3f52aeb204599bc33d8f26d72b11d P b32ae86eec844d9f9a48e020324e4b6f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99339ad25be24f1ebc8448c52077feb7" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 41611 } } peers { permanent_uuid: "ab2829755add43909af910c8aa0b60cf" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 43971 } } peers { permanent_uuid: "b32ae86eec844d9f9a48e020324e4b6f" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 33729 } }
I20260809 02:03:27.863606 13081 catalog_manager.cc:5495] T 9be3f52aeb204599bc33d8f26d72b11d P b32ae86eec844d9f9a48e020324e4b6f reported cstate change: term changed from 0 to 1, leader changed from <none> to b32ae86eec844d9f9a48e020324e4b6f (127.1.192.194). New cstate: current_term: 1 leader_uuid: "b32ae86eec844d9f9a48e020324e4b6f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99339ad25be24f1ebc8448c52077feb7" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 41611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ab2829755add43909af910c8aa0b60cf" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 43971 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b32ae86eec844d9f9a48e020324e4b6f" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 33729 } health_report { overall_health: HEALTHY } } }
I20260809 02:03:27.893357 13602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:27.889650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43446 (local address 127.1.192.194:33729)
0809 02:03:27.889794 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0809 02:03:27.889799 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:03:27.889813 (+ 14us) server_negotiation.cc:407] Connection header received
0809 02:03:27.889894 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:03:27.889896 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:03:27.889971 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:03:27.890073 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:03:27.890396 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:27.890981 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:27.891849 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:03:27.892068 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:03:27.892165 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:03:27.892491 (+ 326us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:03:27.892971 (+ 480us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:03:27.893068 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0809 02:03:27.893186 (+ 118us) server_negotiation.cc:299] Negotiation successful
0809 02:03:27.893242 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 02:03:27.900007 13242 raft_consensus.cc:1261] T 9be3f52aeb204599bc33d8f26d72b11d P ab2829755add43909af910c8aa0b60cf [term 1 FOLLOWER]: Refusing update from remote peer b32ae86eec844d9f9a48e020324e4b6f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:03:27.900002 13538 raft_consensus.cc:1261] T 9be3f52aeb204599bc33d8f26d72b11d P 99339ad25be24f1ebc8448c52077feb7 [term 1 FOLLOWER]: Refusing update from remote peer b32ae86eec844d9f9a48e020324e4b6f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:03:27.900874 13617 consensus_queue.cc:1035] T 9be3f52aeb204599bc33d8f26d72b11d P b32ae86eec844d9f9a48e020324e4b6f [LEADER]: Connected to new peer: Peer: permanent_uuid: "99339ad25be24f1ebc8448c52077feb7" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 41611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:03:27.901124 13612 consensus_queue.cc:1035] T 9be3f52aeb204599bc33d8f26d72b11d P b32ae86eec844d9f9a48e020324e4b6f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ab2829755add43909af910c8aa0b60cf" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 43971 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:03:27.920562 13081 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53196:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:03:27.920729 13081 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53196:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:03:27.923377 13081 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1b0e139fc35b41ae9f8616780994f3c7: Sending DeleteTablet for 3 replicas of tablet 9be3f52aeb204599bc33d8f26d72b11d
I20260809 02:03:27.924240 13516 tablet_service.cc:1514] Processing DeleteTablet for tablet 9be3f52aeb204599bc33d8f26d72b11d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:03:27 UTC) from {username='kudu', principal='kudu/127.1.192.254@KRBTEST.COM'} at 127.0.0.1:46082
I20260809 02:03:27.924269 13220 tablet_service.cc:1514] Processing DeleteTablet for tablet 9be3f52aeb204599bc33d8f26d72b11d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:03:27 UTC) from {username='kudu', principal='kudu/127.1.192.254@KRBTEST.COM'} at 127.0.0.1:46910
I20260809 02:03:27.924391 13368 tablet_service.cc:1514] Processing DeleteTablet for tablet 9be3f52aeb204599bc33d8f26d72b11d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:03:27 UTC) from {username='kudu', principal='kudu/127.1.192.254@KRBTEST.COM'} at 127.0.0.1:43432
I20260809 02:03:27.924822 13635 tablet_replica.cc:330] T 9be3f52aeb204599bc33d8f26d72b11d P b32ae86eec844d9f9a48e020324e4b6f: stopping tablet replica
I20260809 02:03:27.925007 13632 tablet_replica.cc:330] T 9be3f52aeb204599bc33d8f26d72b11d P 99339ad25be24f1ebc8448c52077feb7: stopping tablet replica
I20260809 02:03:27.925161 13635 raft_consensus.cc:2229] T 9be3f52aeb204599bc33d8f26d72b11d P b32ae86eec844d9f9a48e020324e4b6f [term 1 LEADER]: Raft consensus shutting down.
I20260809 02:03:27.925275 13632 raft_consensus.cc:2229] T 9be3f52aeb204599bc33d8f26d72b11d P 99339ad25be24f1ebc8448c52077feb7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:03:27.925719 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 13150
I20260809 02:03:27.931536 13635 raft_consensus.cc:2258] T 9be3f52aeb204599bc33d8f26d72b11d P b32ae86eec844d9f9a48e020324e4b6f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:03:27.925630 13632 raft_consensus.cc:2258] T 9be3f52aeb204599bc33d8f26d72b11d P 99339ad25be24f1ebc8448c52077feb7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:03:27.933104 13635 ts_tablet_manager.cc:1905] T 9be3f52aeb204599bc33d8f26d72b11d P b32ae86eec844d9f9a48e020324e4b6f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:03:27.934425 13632 ts_tablet_manager.cc:1905] T 9be3f52aeb204599bc33d8f26d72b11d P 99339ad25be24f1ebc8448c52077feb7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:03:27.936460 13635 ts_tablet_manager.cc:1918] T 9be3f52aeb204599bc33d8f26d72b11d P b32ae86eec844d9f9a48e020324e4b6f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:03:27.936652 13635 log.cc:1196] T 9be3f52aeb204599bc33d8f26d72b11d P b32ae86eec844d9f9a48e020324e4b6f: Deleting WAL directory at /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786240941968412-1795-0/minicluster-data/ts-1/wal/wals/9be3f52aeb204599bc33d8f26d72b11d
I20260809 02:03:27.937048 13635 ts_tablet_manager.cc:1939] T 9be3f52aeb204599bc33d8f26d72b11d P b32ae86eec844d9f9a48e020324e4b6f: Deleting consensus metadata
I20260809 02:03:27.937952 13632 ts_tablet_manager.cc:1918] T 9be3f52aeb204599bc33d8f26d72b11d P 99339ad25be24f1ebc8448c52077feb7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:03:27.938072 13632 log.cc:1196] T 9be3f52aeb204599bc33d8f26d72b11d P 99339ad25be24f1ebc8448c52077feb7: Deleting WAL directory at /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786240941968412-1795-0/minicluster-data/ts-2/wal/wals/9be3f52aeb204599bc33d8f26d72b11d
W20260809 02:03:27.938373 13070 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.192.193:43971 (error 108)
I20260809 02:03:27.938565 13072 catalog_manager.cc:4841] TS b32ae86eec844d9f9a48e020324e4b6f (127.1.192.194:33729): tablet 9be3f52aeb204599bc33d8f26d72b11d (table test-table [id=09d657a73ae342f581065642ccdea692]) successfully deleted
I20260809 02:03:27.938679 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 13299
I20260809 02:03:27.942617 13632 ts_tablet_manager.cc:1939] T 9be3f52aeb204599bc33d8f26d72b11d P 99339ad25be24f1ebc8448c52077feb7: Deleting consensus metadata
I20260809 02:03:27.943938 13069 catalog_manager.cc:4841] TS 99339ad25be24f1ebc8448c52077feb7 (127.1.192.195:41611): tablet 9be3f52aeb204599bc33d8f26d72b11d (table test-table [id=09d657a73ae342f581065642ccdea692]) successfully deleted
I20260809 02:03:27.944645 13600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:03:27.944356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.193:43971 (local address 127.0.0.1:46916)
0809 02:03:27.944498 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0809 02:03:27.944568 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.193:43971: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260809 02:03:27.944826 13070 catalog_manager.cc:4584] TS ab2829755add43909af910c8aa0b60cf (127.1.192.193:43971): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9be3f52aeb204599bc33d8f26d72b11d: Network error: Client connection negotiation failed: client connection to 127.1.192.193:43971: connect: Connection refused (error 111)
I20260809 02:03:27.947767 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 13448
I20260809 02:03:27.955098 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 13052
2026-08-09T02:03:27Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2720 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:03:28 dist-test-slave-ss4n krb5kdc[13640](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:03:28 dist-test-slave-ss4n krb5kdc[13640](info): set up 2 sockets
Aug 09 02:03:28 dist-test-slave-ss4n krb5kdc[13640](info): commencing operation
krb5kdc: starting...
W20260809 02:03:29.115300 1795 mini_kdc.cc:121] Time spent starting KDC: real 1.134s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/krb5kdc/test-user.keytab.
Aug 09 02:03:29 dist-test-slave-ss4n krb5kdc[13640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241009, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:03:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:03:29Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.1.192.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.1.192.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.1.192.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/krb5kdc/rangeradmin_127.1.192.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.1.192.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/krb5kdc/rangeradmin_127.1.192.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.1.192.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.1.192.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.1.192.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/krb5kdc/rangerlookup_127.1.192.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.1.192.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/krb5kdc/rangerlookup_127.1.192.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.1.192.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.1.192.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.1.192.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/krb5kdc/HTTP_127.1.192.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.1.192.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/krb5kdc/HTTP_127.1.192.212@KRBTEST.COM.keytab.
I20260809 02:03:29.328332 1795 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskRWTcX9/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/minicluster-data/postgres -l logfile start
2026-08-09 02:03:32.291 UTC [13674] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-09 02:03:32.291 UTC [13674] LOG: listening on IPv4 address "127.1.192.212", port 60569
2026-08-09 02:03:32.296 UTC [13674] LOG: listening on Unix socket "/tmp/.s.PGSQL.60569"
2026-08-09 02:03:32.320 UTC [13677] LOG: database system was shut down at 2026-08-09 02:03:30 UTC
2026-08-09 02:03:32.324 UTC [13674] LOG: database system is ready to accept connections
I20260809 02:03:33.365291 1795 mini_postgres.cc:96] Postgres bound to 60569
2026-08-09 02:03:33.373 UTC [13686] FATAL: database "slave" does not exist
127.1.192.212:60569 - accepting connections
I20260809 02:03:33.374892 1795 mini_ranger.cc:161] Starting Ranger...
I20260809 02:03:33.394026 1795 mini_ranger.cc:84] Created miniranger Postgres user
I20260809 02:03:33.984268 1795 mini_ranger.cc:87] Created ranger Postgres database
I20260809 02:03:33.984395 1795 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/minicluster-data/ranger-admin
2026-08-09 02:03:34,325 [I] DB FLAVOR :POSTGRES
2026-08-09 02:03:34,326 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 02:03:34,326 [I] Checking connection..
2026-08-09 02:03:34,326 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRWTcX9/build/debug/bin/postgresql.jar:/tmp/dist-test-taskRWTcX9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.192.212:60569/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-09 02:03:34,608 [I] Checking connection passed.
2026-08-09 02:03:34,608 [I] --------- Verifying version history table ---------
2026-08-09 02:03:34,609 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRWTcX9/build/debug/bin/postgresql.jar:/tmp/dist-test-taskRWTcX9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.192.212:60569/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 02:03:34,879 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 02:03:34,879 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRWTcX9/build/debug/bin/postgresql.jar:/tmp/dist-test-taskRWTcX9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.192.212:60569/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 02:03:35,153 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 02:03:35,153 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-09 02:03:35,153 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRWTcX9/build/debug/bin/postgresql.jar:/tmp/dist-test-taskRWTcX9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.192.212:60569/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskRWTcX9/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-09 02:03:35.434 UTC [13752] WARNING: there is no transaction in progress
2026-08-09 02:03:35,442 [I] create_dbversion_catalog.sql file imported successfully
2026-08-09 02:03:35,442 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRWTcX9/build/debug/bin/postgresql.jar:/tmp/dist-test-taskRWTcX9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.192.212:60569/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 02:03:35,700 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-09 02:03:35,701 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 02:03:35,701 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRWTcX9/build/debug/bin/postgresql.jar:/tmp/dist-test-taskRWTcX9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.192.212:60569/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-09 02:03:35,948 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRWTcX9/build/debug/bin/postgresql.jar:/tmp/dist-test-taskRWTcX9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.192.212:60569/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-09 02:03:36,205 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRWTcX9/build/debug/bin/postgresql.jar:/tmp/dist-test-taskRWTcX9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.192.212:60569/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-ss4n.c.gcp-upstream.internal','N') ;"
2026-08-09 02:03:36,457 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRWTcX9/build/debug/bin/postgresql.jar:/tmp/dist-test-taskRWTcX9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.192.212:60569/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 02:03:36,704 [I] Table x_portal_user does not exist in database ranger
2026-08-09 02:03:36,705 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRWTcX9/build/debug/bin/postgresql.jar:/tmp/dist-test-taskRWTcX9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.192.212:60569/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 02:03:36,989 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-09 02:03:36,990 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRWTcX9/build/debug/bin/postgresql.jar:/tmp/dist-test-taskRWTcX9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.192.212:60569/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 02:03:37,264 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRWTcX9/build/debug/bin/postgresql.jar:/tmp/dist-test-taskRWTcX9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.192.212:60569/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 02:03:37,528 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-09 02:03:37,528 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRWTcX9/build/debug/bin/postgresql.jar:/tmp/dist-test-taskRWTcX9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.192.212:60569/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskRWTcX9/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-09 02:03:38.596 UTC [13871] WARNING: there is no transaction in progress
2026-08-09 02:03:38.615 UTC [13871] WARNING: there is no transaction in progress
2026-08-09 02:03:38.635 UTC [13871] WARNING: there is no transaction in progress
2026-08-09 02:03:38.654 UTC [13871] WARNING: there is no transaction in progress
2026-08-09 02:03:38.674 UTC [13871] WARNING: there is no transaction in progress
2026-08-09 02:03:38.694 UTC [13871] WARNING: there is no transaction in progress
2026-08-09 02:03:38.758 UTC [13871] WARNING: there is no transaction in progress
2026-08-09 02:03:38.767 UTC [13871] WARNING: there is no transaction in progress
2026-08-09 02:03:38.784 UTC [13871] WARNING: there is no transaction in progress
2026-08-09 02:03:38.797 UTC [13871] WARNING: there is no transaction in progress
2026-08-09 02:03:38.809 UTC [13871] WARNING: there is no transaction in progress
2026-08-09 02:03:38.827 UTC [13871] WARNING: there is no transaction in progress
2026-08-09 02:03:38.839 UTC [13871] WARNING: there is no transaction in progress
2026-08-09 02:03:38.847 UTC [13871] WARNING: there is no transaction in progress
2026-08-09 02:03:38.855 UTC [13871] WARNING: there is no transaction in progress
2026-08-09 02:03:38.863 UTC [13871] WARNING: there is no transaction in progress
2026-08-09 02:03:39,624 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-09 02:03:39,625 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRWTcX9/build/debug/bin/postgresql.jar:/tmp/dist-test-taskRWTcX9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.192.212:60569/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-09 02:03:39,893 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-09 02:03:39,894 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRWTcX9/build/debug/bin/postgresql.jar:/tmp/dist-test-taskRWTcX9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.192.212:60569/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 02:03:40,175 [I] Table x_portal_user already exists in database 'ranger'
2026-08-09 02:03:40,176 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRWTcX9/build/debug/bin/postgresql.jar:/tmp/dist-test-taskRWTcX9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.192.212:60569/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 02:03:40,477 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-09 02:03:40,477 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRWTcX9/build/debug/bin/postgresql.jar:/tmp/dist-test-taskRWTcX9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.192.212:60569/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 02:03:40,739 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRWTcX9/build/debug/bin/postgresql.jar:/tmp/dist-test-taskRWTcX9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.192.212:60569/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 02:03:40,977 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRWTcX9/build/debug/bin/postgresql.jar:/tmp/dist-test-taskRWTcX9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.192.212:60569/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-ss4n.c.gcp-upstream.internal';"
2026-08-09 02:03:41,227 [I] CORE_DB_SCHEMA import status has been updated
2026-08-09 02:03:41,227 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRWTcX9/build/debug/bin/postgresql.jar:/tmp/dist-test-taskRWTcX9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.192.212:60569/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-09 02:03:41,468 [I] DB_PATCHES have already been applied
I20260809 02:03:41.474058 1795 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/minicluster-data/ranger-admin:/tmp/dist-test-taskRWTcX9/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskRWTcX9/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskRWTcX9/build/debug/bin/postgresql.jar:/tmp/dist-test-taskRWTcX9/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260809 02:03:41.474162 1795 mini_ranger.cc:193] Using host: 127.1.192.212
I20260809 02:03:41.478437 1795 mini_ranger.cc:227] Ranger admin URL: http://127.1.192.212:59907
Aug 09, 2026 2:03:41 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 2:03:41 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 2:03:41 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/minicluster-data/ranger-admin/ews/webapp
Aug 09, 2026 2:03:41 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 09, 2026 2:03:41 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 09, 2026 2:03:42 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/minicluster-data/ranger-admin/ews/webapp].
Aug 09, 2026 2:03:42 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260809 02:03:42.632236 1795 mini_ranger.cc:160] Time spent starting Ranger: real 9.257s user 0.010s sys 0.004s
Aug 09, 2026 2:04:04 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 09, 2026 2:04:04 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 09, 2026 2:04:04 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 09, 2026 2:04:04 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 09, 2026 2:04:04 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 09, 2026 2:04:04 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 09, 2026 2:04:04 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 09, 2026 2:04:04 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 09, 2026 2:04:04 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 09, 2026 2:04:04 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 09, 2026 2:04:04 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 09, 2026 2:04:04 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 09, 2026 2:04:04 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 09, 2026 2:04:04 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 09, 2026 2:04:04 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 09, 2026 2:04:04 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 09, 2026 2:04:04 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 09, 2026 2:04:05 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260809 02:04:06.446200 1795 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.1.192.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.1.192.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.1.192.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/krb5kdc/rangerkms_127.1.192.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.1.192.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/krb5kdc/rangerkms_127.1.192.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.1.192.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.1.192.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.1.192.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/krb5kdc/HTTP_127.1.192.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.1.192.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/krb5kdc/HTTP_127.1.192.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 09 02:04:06 dist-test-slave-ss4n krb5kdc[13640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241046, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260809 02:04:06.543859 1795 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260809 02:04:06.554807 1795 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260809 02:04:07.561618 1795 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260809 02:04:07.561725 1795 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/minicluster-data/ranger-kms
I20260809 02:04:07.561849 1795 mini_ranger_kms.cc:227] Using postgres at 127.1.192.212:60569
2026-08-09 02:04:07,805 [I] DB FLAVOR :POSTGRES
2026-08-09 02:04:07,805 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 02:04:07,805 [I] Checking connection
2026-08-09 02:04:07,805 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRWTcX9/build/debug/bin/postgresql.jar:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.192.212:60569/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-09 02:04:08,084 [I] connection success
2026-08-09 02:04:08,085 [I] --------- Verifying Ranger DB tables ---------
2026-08-09 02:04:08,085 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-09 02:04:08,085 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRWTcX9/build/debug/bin/postgresql.jar:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.192.212:60569/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-09 02:04:08,404 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-09 02:04:08,404 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 02:04:08,404 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-09 02:04:08,404 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRWTcX9/build/debug/bin/postgresql.jar:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.192.212:60569/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-09 02:04:08,692 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260809 02:04:09.180260 1795 mini_ranger_kms.cc:310] Created kms service
I20260809 02:04:09.400748 1795 mini_ranger_kms.cc:326] Created kudu user
I20260809 02:04:09.467381 1795 mini_ranger_kms.cc:343] Created rangerkms user
Aug 09 02:04:09 dist-test-slave-ss4n krb5kdc[13640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241046, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.1.192.212@KRBTEST.COM
I20260809 02:04:09.881349 1795 mini_ranger_kms.cc:384] Added ranger policy
I20260809 02:04:09.881696 1795 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/minicluster-data/ranger-kms:/tmp/dist-test-taskRWTcX9/build/debug/bin/postgresql.jar:/tmp/dist-test-taskRWTcX9/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskRWTcX9/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260809 02:04:09.881745 1795 mini_ranger_kms.cc:240] Using host: 127.1.192.212
I20260809 02:04:09.886345 1795 mini_ranger_kms.cc:276] Ranger KMS PID: 14066
I20260809 02:04:09.886488 1795 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.1.192.212:37655
Aug 09, 2026 2:04:10 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 2:04:10 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 2:04:11 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 09, 2026 2:04:11 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 09, 2026 2:04:11 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/minicluster-data/ranger-kms/ews/webapp].
Aug 09, 2026 2:04:11 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260809 02:04:12.373703 1795 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.830s user 0.005s sys 0.220s
I20260809 02:04:12.373826 1795 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260809 02:04:12.373987 1795 mini_ranger_kms.cc:401] 127.1.192.212:37655/kms/v1/keys
Aug 09 02:04:12 dist-test-slave-ss4n krb5kdc[13640](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786241052, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.192.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 09 02:04:13 dist-test-slave-ss4n krb5kdc[13640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241052, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.192.212@KRBTEST.COM for HTTP/127.1.192.212@KRBTEST.COM
Aug 09 02:04:13 dist-test-slave-ss4n krb5kdc[13640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241052, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.192.212@KRBTEST.COM for HTTP/127.1.192.212@KRBTEST.COM
Aug 09 02:04:15 dist-test-slave-ss4n krb5kdc[13640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241055, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.1.192.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.192.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.192.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab.
Entry for principal kudu/127.1.192.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab.
I20260809 02:04:15.894265 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.192.254:46009
--webserver_interface=127.1.192.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab
--principal=kudu/127.1.192.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:45947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.192.254:46009
--ranger_config_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.192.212:37655/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:04:16.029636 14120 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:04:16.029942 14120 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:04:16.029982 14120 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:04:16.034305 14120 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:04:16.034382 14120 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:04:16.034402 14120 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:04:16.034418 14120 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:04:16.034435 14120 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:04:16.040285 14120 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:45947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/minicluster-data/master-0/wal
--ranger_kms_url=127.1.192.212:37655/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.1.192.254:46009
--ranger_config_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab
--principal=kudu/127.1.192.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.192.254:46009
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.192.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14120
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:04:16.041741 14120 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:04:16.042865 14120 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:04:16.052136 14125 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:04:16.052268 14120 server_base.cc:1034] running on GCE node
W20260809 02:04:16.052407 14126 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:04:16.052484 14128 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:04:16.053092 14120 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:04:16.054443 14120 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:04:16.055648 14120 hybrid_clock.cc:648] HybridClock initialized: now 1786241056055630 us; error 38 us; skew 500 ppm
Aug 09 02:04:16 dist-test-slave-ss4n krb5kdc[13640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241056, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:04:16.064270 14120 init.cc:377] Logged in from keytab as kudu/127.1.192.254@KRBTEST.COM (short username kudu)
I20260809 02:04:16.065626 14120 webserver.cc:460] Webserver started at http://127.1.192.254:41479/ using document root <none> and password file <none>
I20260809 02:04:16.066308 14120 fs_manager.cc:359] Metadata directory not provided
I20260809 02:04:16.066406 14120 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:04:16.066689 14120 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 02:04:16 dist-test-slave-ss4n krb5kdc[13640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241056, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.254@KRBTEST.COM for HTTP/127.1.192.212@KRBTEST.COM
I20260809 02:04:16.123056 14120 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/minicluster-data/master-0/data/instance:
uuid: "f4210c06d2744ebc88b4288e24f62180"
format_stamp: "Formatted at 2026-08-09 02:04:16 on dist-test-slave-ss4n"
server_key: "b1b35ee471829d29c5425dc68665611a"
server_key_iv: "62a3b66b4c60f4f03c32f9bece248dcf"
server_key_version: "kuduclusterkey@0"
I20260809 02:04:16.123972 14120 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/minicluster-data/master-0/wal/instance:
uuid: "f4210c06d2744ebc88b4288e24f62180"
format_stamp: "Formatted at 2026-08-09 02:04:16 on dist-test-slave-ss4n"
server_key: "b1b35ee471829d29c5425dc68665611a"
server_key_iv: "62a3b66b4c60f4f03c32f9bece248dcf"
server_key_version: "kuduclusterkey@0"
I20260809 02:04:16.129274 14120 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260809 02:04:16.144896 14135 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:04:16.146426 14120 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 02:04:16.146595 14120 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/minicluster-data/master-0/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/minicluster-data/master-0/wal
uuid: "f4210c06d2744ebc88b4288e24f62180"
format_stamp: "Formatted at 2026-08-09 02:04:16 on dist-test-slave-ss4n"
server_key: "b1b35ee471829d29c5425dc68665611a"
server_key_iv: "62a3b66b4c60f4f03c32f9bece248dcf"
server_key_version: "kuduclusterkey@0"
I20260809 02:04:16.146759 14120 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:04:16.168959 14120 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:04:16.172531 14120 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:04:16.172771 14120 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:04:16.181046 14120 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.254:46009
I20260809 02:04:16.181082 14191 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.254:46009 every 8 connection(s)
I20260809 02:04:16.182241 14120 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
I20260809 02:04:16.184346 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 14120
I20260809 02:04:16.184477 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/minicluster-data/master-0/wal/instance
I20260809 02:04:16.184793 1795 external_mini_cluster.cc:1428] Setting key 9b9974ce5ba8b703ef6877ecac4f4b30
I20260809 02:04:16.185454 14192 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:04:16.193385 14192 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f4210c06d2744ebc88b4288e24f62180: Bootstrap starting.
I20260809 02:04:16.196198 14192 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f4210c06d2744ebc88b4288e24f62180: Neither blocks nor log segments found. Creating new log.
Aug 09 02:04:16 dist-test-slave-ss4n krb5kdc[13640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241055, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.192.254@KRBTEST.COM
I20260809 02:04:16.197274 14192 log.cc:824] T 00000000000000000000000000000000 P f4210c06d2744ebc88b4288e24f62180: Log is configured to *not* fsync() on all Append() calls
I20260809 02:04:16.199859 14192 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f4210c06d2744ebc88b4288e24f62180: No bootstrap required, opened a new log
I20260809 02:04:16.204121 14195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:04:16.186562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56140 (local address 127.1.192.254:46009)
0809 02:04:16.187178 (+ 616us) server_negotiation.cc:206] Beginning negotiation
0809 02:04:16.187199 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0809 02:04:16.187339 (+ 140us) server_negotiation.cc:407] Connection header received
0809 02:04:16.188510 (+ 1171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:04:16.188534 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:04:16.188968 (+ 434us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:04:16.189357 (+ 389us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:04:16.190351 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:04:16.192108 (+ 1757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:04:16.193402 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:04:16.194143 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:04:16.194325 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:04:16.197711 (+ 3386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:04:16.197718 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:04:16.197736 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:04:16.197772 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:04:16.200808 (+ 3036us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:04:16.201360 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:04:16.201366 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:04:16.201371 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:04:16.201556 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:04:16.201890 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:04:16.201894 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:04:16.201896 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:04:16.202369 (+ 473us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:04:16.202592 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0809 02:04:16.202952 (+ 360us) server_negotiation.cc:299] Negotiation successful
0809 02:04:16.203307 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":449,"thread_start_us":163,"threads_started":1}
I20260809 02:04:16.204134 14192 raft_consensus.cc:348] T 00000000000000000000000000000000 P f4210c06d2744ebc88b4288e24f62180 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4210c06d2744ebc88b4288e24f62180" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 46009 } }
I20260809 02:04:16.204495 14192 raft_consensus.cc:374] T 00000000000000000000000000000000 P f4210c06d2744ebc88b4288e24f62180 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:04:16.204569 14192 raft_consensus.cc:729] T 00000000000000000000000000000000 P f4210c06d2744ebc88b4288e24f62180 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4210c06d2744ebc88b4288e24f62180, State: Initialized, Role: FOLLOWER
I20260809 02:04:16.205077 14192 consensus_queue.cc:260] T 00000000000000000000000000000000 P f4210c06d2744ebc88b4288e24f62180 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4210c06d2744ebc88b4288e24f62180" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 46009 } }
I20260809 02:04:16.205245 14192 raft_consensus.cc:388] T 00000000000000000000000000000000 P f4210c06d2744ebc88b4288e24f62180 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:04:16.205303 14192 raft_consensus.cc:482] T 00000000000000000000000000000000 P f4210c06d2744ebc88b4288e24f62180 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:04:16.205394 14192 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f4210c06d2744ebc88b4288e24f62180 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:04:16.206650 14192 raft_consensus.cc:504] T 00000000000000000000000000000000 P f4210c06d2744ebc88b4288e24f62180 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4210c06d2744ebc88b4288e24f62180" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 46009 } }
I20260809 02:04:16.207012 14192 leader_election.cc:302] T 00000000000000000000000000000000 P f4210c06d2744ebc88b4288e24f62180 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f4210c06d2744ebc88b4288e24f62180; no voters:
I20260809 02:04:16.207422 14192 leader_election.cc:288] T 00000000000000000000000000000000 P f4210c06d2744ebc88b4288e24f62180 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:04:16.207607 14197 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f4210c06d2744ebc88b4288e24f62180 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:04:16.207890 14197 raft_consensus.cc:686] T 00000000000000000000000000000000 P f4210c06d2744ebc88b4288e24f62180 [term 1 LEADER]: Becoming Leader. State: Replica: f4210c06d2744ebc88b4288e24f62180, State: Running, Role: LEADER
I20260809 02:04:16.208276 14197 consensus_queue.cc:237] T 00000000000000000000000000000000 P f4210c06d2744ebc88b4288e24f62180 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4210c06d2744ebc88b4288e24f62180" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 46009 } }
I20260809 02:04:16.208812 14192 sys_catalog.cc:564] T 00000000000000000000000000000000 P f4210c06d2744ebc88b4288e24f62180 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:04:16.209635 14197 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4210c06d2744ebc88b4288e24f62180 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f4210c06d2744ebc88b4288e24f62180. Latest consensus state: current_term: 1 leader_uuid: "f4210c06d2744ebc88b4288e24f62180" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4210c06d2744ebc88b4288e24f62180" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 46009 } } }
I20260809 02:04:16.209949 14197 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4210c06d2744ebc88b4288e24f62180 [sys.catalog]: This master's current role is: LEADER
I20260809 02:04:16.210166 14199 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4210c06d2744ebc88b4288e24f62180 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f4210c06d2744ebc88b4288e24f62180" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4210c06d2744ebc88b4288e24f62180" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 46009 } } }
I20260809 02:04:16.210268 14199 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4210c06d2744ebc88b4288e24f62180 [sys.catalog]: This master's current role is: LEADER
I20260809 02:04:16.210521 14207 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:04:16.214514 14207 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:04:16.215124 14192 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260809 02:04:16.232275 14207 catalog_manager.cc:1347] Generated new cluster ID: f700a9955e8545488f9c98c3987bdab1
I20260809 02:04:16.232378 14207 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:04:16.255946 14207 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:04:16.257596 14207 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:04:16.263130 14207 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f4210c06d2744ebc88b4288e24f62180: Generated new TSK 0
I20260809 02:04:16.264183 14207 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.192.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.192.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.192.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.193.keytab.
Entry for principal kudu/127.1.192.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.193.keytab.
I20260809 02:04:17.053859 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.192.212:37655/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.192.193:0
--local_ip_for_outbound_sockets=127.1.192.193
--webserver_interface=127.1.192.193
--webserver_port=0
--tserver_master_addrs=127.1.192.254:46009
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.193.keytab
--principal=kudu/127.1.192.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:45947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:04:17.225042 14241 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:04:17.225451 14241 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:04:17.225548 14241 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:04:17.231896 14241 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:04:17.231997 14241 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:04:17.232095 14241 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.192.193
I20260809 02:04:17.240597 14241 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:45947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.1.192.212:37655/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.193.keytab
--principal=kudu/127.1.192.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.192.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.192.193
--webserver_port=0
--tserver_master_addrs=127.1.192.254:46009
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14241
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.192.193
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:04:17.242151 14241 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:04:17.243549 14241 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:04:17.254357 14249 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:04:17.254743 14241 server_base.cc:1034] running on GCE node
W20260809 02:04:17.254914 14248 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:04:17.254369 14251 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:04:17.255633 14241 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:04:17.256503 14241 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:04:17.257696 14241 hybrid_clock.cc:648] HybridClock initialized: now 1786241057257669 us; error 43 us; skew 500 ppm
Aug 09 02:04:17 dist-test-slave-ss4n krb5kdc[13640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241057, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:04:17.261567 14241 init.cc:377] Logged in from keytab as kudu/127.1.192.193@KRBTEST.COM (short username kudu)
I20260809 02:04:17.262848 14241 webserver.cc:460] Webserver started at http://127.1.192.193:36457/ using document root <none> and password file <none>
I20260809 02:04:17.263679 14241 fs_manager.cc:359] Metadata directory not provided
I20260809 02:04:17.263793 14241 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:04:17.264127 14241 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 02:04:17 dist-test-slave-ss4n krb5kdc[13640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241057, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.193@KRBTEST.COM for HTTP/127.1.192.212@KRBTEST.COM
I20260809 02:04:17.280431 14241 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/minicluster-data/ts-0/data/instance:
uuid: "23a14f53e268416f816e5a1f62b3605d"
format_stamp: "Formatted at 2026-08-09 02:04:17 on dist-test-slave-ss4n"
server_key: "a5e5458aabf1c4631b3777784d4b51ac"
server_key_iv: "0bb8f35a89ec24d73a58560e5fd0b63a"
server_key_version: "kuduclusterkey@0"
I20260809 02:04:17.281230 14241 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/minicluster-data/ts-0/wal/instance:
uuid: "23a14f53e268416f816e5a1f62b3605d"
format_stamp: "Formatted at 2026-08-09 02:04:17 on dist-test-slave-ss4n"
server_key: "a5e5458aabf1c4631b3777784d4b51ac"
server_key_iv: "0bb8f35a89ec24d73a58560e5fd0b63a"
server_key_version: "kuduclusterkey@0"
I20260809 02:04:17.286765 14241 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.000s
I20260809 02:04:17.325472 14259 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:04:17.328544 14241 fs_manager.cc:725] Time spent opening block manager: real 0.011s user 0.002s sys 0.000s
I20260809 02:04:17.328718 14241 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/minicluster-data/ts-0/wal
uuid: "23a14f53e268416f816e5a1f62b3605d"
format_stamp: "Formatted at 2026-08-09 02:04:17 on dist-test-slave-ss4n"
server_key: "a5e5458aabf1c4631b3777784d4b51ac"
server_key_iv: "0bb8f35a89ec24d73a58560e5fd0b63a"
server_key_version: "kuduclusterkey@0"
I20260809 02:04:17.328874 14241 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:04:17.363577 14241 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:04:17.366961 14241 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:04:17.367192 14241 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:04:17.367949 14241 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:04:17.368880 14241 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:04:17.368950 14241 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:04:17.369050 14241 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:04:17.369112 14241 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:04:17.380620 14241 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.193:36437
I20260809 02:04:17.380702 14382 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.193:36437 every 8 connection(s)
I20260809 02:04:17.381740 14241 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/minicluster-data/ts-0/data/info.pb
I20260809 02:04:17.387985 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 14241
I20260809 02:04:17.388115 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/minicluster-data/ts-0/wal/instance
I20260809 02:04:17.388432 1795 external_mini_cluster.cc:1428] Setting key 8fcf6fa081dbee49311d5d5267617b86
Aug 09 02:04:17 dist-test-slave-ss4n krb5kdc[13640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241057, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.193@KRBTEST.COM for kudu/127.1.192.254@KRBTEST.COM
I20260809 02:04:17.400135 14386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:04:17.384251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:46009 (local address 127.1.192.193:53495)
0809 02:04:17.384883 (+ 632us) negotiation.cc:107] Waiting for socket to connect
0809 02:04:17.384964 (+ 81us) client_negotiation.cc:174] Beginning negotiation
0809 02:04:17.385957 (+ 993us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:04:17.386537 (+ 580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:04:17.386544 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:04:17.386825 (+ 281us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:04:17.387306 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:04:17.387320 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:04:17.388393 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:04:17.388397 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:04:17.390331 (+ 1934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:04:17.390345 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:04:17.391185 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:04:17.391192 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:04:17.391429 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:04:17.392187 (+ 758us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:04:17.392216 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:04:17.394110 (+ 1894us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:04:17.396327 (+ 2217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:04:17.396337 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:04:17.396358 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:04:17.396748 (+ 390us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:04:17.397048 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:04:17.397054 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:04:17.397058 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:04:17.397267 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:04:17.397868 (+ 601us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:04:17.397878 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:04:17.398268 (+ 390us) client_negotiation.cc:769] Sending connection context
0809 02:04:17.398494 (+ 226us) client_negotiation.cc:240] Negotiation successful
0809 02:04:17.398875 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":187,"threads_started":1}
I20260809 02:04:17.401121 14385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:04:17.383809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.193:53495 (local address 127.1.192.254:46009)
0809 02:04:17.385334 (+ 1525us) server_negotiation.cc:206] Beginning negotiation
0809 02:04:17.385339 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:04:17.385357 (+ 18us) server_negotiation.cc:407] Connection header received
0809 02:04:17.386174 (+ 817us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:04:17.386177 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:04:17.386238 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:04:17.386354 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:04:17.387458 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:04:17.388226 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:04:17.390477 (+ 2251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:04:17.390857 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:04:17.390974 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:04:17.394346 (+ 3372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:04:17.394350 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:04:17.394353 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:04:17.394387 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:04:17.396194 (+ 1807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:04:17.396854 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:04:17.396859 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:04:17.396861 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:04:17.396951 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:04:17.397384 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:04:17.397390 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:04:17.397395 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:04:17.397700 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:04:17.397800 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0809 02:04:17.400751 (+ 2951us) server_negotiation.cc:299] Negotiation successful
0809 02:04:17.400906 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1439,"thread_start_us":71,"threads_started":1}
I20260809 02:04:17.402413 14383 heartbeater.cc:347] Connected to a master server at 127.1.192.254:46009
I20260809 02:04:17.402669 14383 heartbeater.cc:464] Registering TS with master...
I20260809 02:04:17.403195 14383 heartbeater.cc:511] Master 127.1.192.254:46009 requested a full tablet report, sending...
I20260809 02:04:17.405021 14154 ts_manager.cc:194] Registered new tserver with Master: 23a14f53e268416f816e5a1f62b3605d (127.1.192.193:36437)
WARNING: no policy specified for kudu/127.1.192.194@KRBTEST.COM; defaulting to no policy
I20260809 02:04:17.406522 14154 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.192.193@KRBTEST.COM'} at 127.1.192.193:53495
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.192.194@KRBTEST.COM" created.
Aug 09 02:04:17 dist-test-slave-ss4n krb5kdc[13640](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786241057, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.192.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.194.keytab.
Entry for principal kudu/127.1.192.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.194.keytab.
I20260809 02:04:17.430455 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.192.212:37655/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.192.194:0
--local_ip_for_outbound_sockets=127.1.192.194
--webserver_interface=127.1.192.194
--webserver_port=0
--tserver_master_addrs=127.1.192.254:46009
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.194.keytab
--principal=kudu/127.1.192.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:45947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:04:17.591665 14389 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:04:17.592175 14389 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:04:17.592307 14389 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:04:17.598852 14389 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:04:17.599027 14389 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:04:17.599174 14389 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.192.194
I20260809 02:04:17.607373 14389 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:45947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.1.192.212:37655/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.194.keytab
--principal=kudu/127.1.192.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.192.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.192.194
--webserver_port=0
--tserver_master_addrs=127.1.192.254:46009
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14389
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.192.194
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:04:17.608778 14389 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:04:17.610132 14389 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:04:17.622363 14398 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:04:17.622766 14389 server_base.cc:1034] running on GCE node
W20260809 02:04:17.622934 14396 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:04:17.623145 14395 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:04:17.623710 14389 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:04:17.624483 14389 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:04:17.625772 14389 hybrid_clock.cc:648] HybridClock initialized: now 1786241057625728 us; error 63 us; skew 500 ppm
Aug 09 02:04:17 dist-test-slave-ss4n krb5kdc[13640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241057, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:04:17.629750 14389 init.cc:377] Logged in from keytab as kudu/127.1.192.194@KRBTEST.COM (short username kudu)
I20260809 02:04:17.631108 14389 webserver.cc:460] Webserver started at http://127.1.192.194:41673/ using document root <none> and password file <none>
I20260809 02:04:17.631945 14389 fs_manager.cc:359] Metadata directory not provided
I20260809 02:04:17.632071 14389 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:04:17.632364 14389 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 02:04:17 dist-test-slave-ss4n krb5kdc[13640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241057, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.194@KRBTEST.COM for HTTP/127.1.192.212@KRBTEST.COM
I20260809 02:04:17.652359 14389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/minicluster-data/ts-1/data/instance:
uuid: "0e0a5584605d40f39f6161b2dc18ec1f"
format_stamp: "Formatted at 2026-08-09 02:04:17 on dist-test-slave-ss4n"
server_key: "e5cf0b3c2e36de215abebb1f391043b6"
server_key_iv: "98fe4cfd00c8d6db8f933a9d2ab27578"
server_key_version: "kuduclusterkey@0"
I20260809 02:04:17.653365 14389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/minicluster-data/ts-1/wal/instance:
uuid: "0e0a5584605d40f39f6161b2dc18ec1f"
format_stamp: "Formatted at 2026-08-09 02:04:17 on dist-test-slave-ss4n"
server_key: "e5cf0b3c2e36de215abebb1f391043b6"
server_key_iv: "98fe4cfd00c8d6db8f933a9d2ab27578"
server_key_version: "kuduclusterkey@0"
I20260809 02:04:17.672785 14389 fs_manager.cc:691] Time spent creating directory manager: real 0.019s user 0.005s sys 0.000s
I20260809 02:04:17.695029 14405 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:04:17.696798 14389 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260809 02:04:17.696941 14389 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/minicluster-data/ts-1/wal
uuid: "0e0a5584605d40f39f6161b2dc18ec1f"
format_stamp: "Formatted at 2026-08-09 02:04:17 on dist-test-slave-ss4n"
server_key: "e5cf0b3c2e36de215abebb1f391043b6"
server_key_iv: "98fe4cfd00c8d6db8f933a9d2ab27578"
server_key_version: "kuduclusterkey@0"
I20260809 02:04:17.697074 14389 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:04:17.712965 14389 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:04:17.716480 14389 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:04:17.716719 14389 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:04:17.717377 14389 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:04:17.718354 14389 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:04:17.718456 14389 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:04:17.718560 14389 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:04:17.718613 14389 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:04:17.730691 14389 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.194:45489
I20260809 02:04:17.731918 14389 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/minicluster-data/ts-1/data/info.pb
I20260809 02:04:17.732771 14526 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.194:45489 every 8 connection(s)
I20260809 02:04:17.732807 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 14389
I20260809 02:04:17.732929 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/minicluster-data/ts-1/wal/instance
I20260809 02:04:17.733244 1795 external_mini_cluster.cc:1428] Setting key cfe52116041cf40b70949135133a699c
Aug 09 02:04:17 dist-test-slave-ss4n krb5kdc[13640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241057, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.194@KRBTEST.COM for kudu/127.1.192.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.192.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.192.195@KRBTEST.COM" created.
I20260809 02:04:17.754530 14385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:04:17.740454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.194:52489 (local address 127.1.192.254:46009)
0809 02:04:17.740598 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0809 02:04:17.740603 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:04:17.740623 (+ 20us) server_negotiation.cc:407] Connection header received
0809 02:04:17.741714 (+ 1091us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:04:17.741718 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:04:17.741775 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:04:17.741833 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:04:17.743579 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:04:17.744427 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:04:17.746202 (+ 1775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:04:17.746541 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:04:17.746622 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:04:17.749816 (+ 3194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:04:17.749820 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:04:17.749823 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:04:17.749854 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:04:17.751854 (+ 2000us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:04:17.752547 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:04:17.752551 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:04:17.752553 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:04:17.752648 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:04:17.753082 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:04:17.753085 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:04:17.753087 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:04:17.753316 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:04:17.753404 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 02:04:17.754181 (+ 777us) server_negotiation.cc:299] Negotiation successful
0809 02:04:17.754352 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 02:04:17.756381 14530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:04:17.739424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:46009 (local address 127.1.192.194:52489)
0809 02:04:17.740072 (+ 648us) negotiation.cc:107] Waiting for socket to connect
0809 02:04:17.740139 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0809 02:04:17.741411 (+ 1272us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:04:17.742303 (+ 892us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:04:17.742313 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:04:17.742687 (+ 374us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:04:17.743384 (+ 697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:04:17.743406 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:04:17.744607 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:04:17.744610 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:04:17.746037 (+ 1427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:04:17.746051 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:04:17.746837 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:04:17.746840 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:04:17.747058 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:04:17.747817 (+ 759us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:04:17.747838 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:04:17.749604 (+ 1766us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:04:17.752013 (+ 2409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:04:17.752020 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:04:17.752039 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:04:17.752416 (+ 377us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:04:17.752765 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:04:17.752768 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:04:17.752770 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:04:17.752956 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:04:17.753501 (+ 545us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:04:17.753510 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:04:17.753859 (+ 349us) client_negotiation.cc:769] Sending connection context
0809 02:04:17.754781 (+ 922us) client_negotiation.cc:240] Negotiation successful
0809 02:04:17.755235 (+ 454us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":170,"threads_started":1}
I20260809 02:04:17.757959 14527 heartbeater.cc:347] Connected to a master server at 127.1.192.254:46009
I20260809 02:04:17.758199 14527 heartbeater.cc:464] Registering TS with master...
I20260809 02:04:17.758757 14527 heartbeater.cc:511] Master 127.1.192.254:46009 requested a full tablet report, sending...
I20260809 02:04:17.759912 14154 ts_manager.cc:194] Registered new tserver with Master: 0e0a5584605d40f39f6161b2dc18ec1f (127.1.192.194:45489)
I20260809 02:04:17.760712 14154 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.192.194@KRBTEST.COM'} at 127.1.192.194:52489
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.192.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.195.keytab.
Entry for principal kudu/127.1.192.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.195.keytab.
I20260809 02:04:17.781330 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.192.212:37655/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.192.195:0
--local_ip_for_outbound_sockets=127.1.192.195
--webserver_interface=127.1.192.195
--webserver_port=0
--tserver_master_addrs=127.1.192.254:46009
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.195.keytab
--principal=kudu/127.1.192.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:45947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 02:04:17 dist-test-slave-ss4n krb5kdc[13640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241057, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.254@KRBTEST.COM for HTTP/127.1.192.212@KRBTEST.COM
W20260809 02:04:17.952313 14532 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:04:17.952698 14532 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:04:17.952775 14532 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:04:17.958762 14532 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:04:17.958856 14532 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:04:17.958941 14532 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.192.195
I20260809 02:04:17.966611 14532 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:45947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.1.192.212:37655/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.195.keytab
--principal=kudu/127.1.192.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.192.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.192.195
--webserver_port=0
--tserver_master_addrs=127.1.192.254:46009
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14532
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.192.195
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:04:17.968201 14532 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:04:17.969573 14532 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:04:17.982261 14540 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:04:17.982307 14537 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:04:17.982613 14538 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:04:17.983206 14532 server_base.cc:1034] running on GCE node
I20260809 02:04:17.986189 14532 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:04:17.986961 14532 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:04:17.988158 14532 hybrid_clock.cc:648] HybridClock initialized: now 1786241057988094 us; error 81 us; skew 500 ppm
Aug 09 02:04:17 dist-test-slave-ss4n krb5kdc[13640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241057, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:04:17.991693 14532 init.cc:377] Logged in from keytab as kudu/127.1.192.195@KRBTEST.COM (short username kudu)
I20260809 02:04:17.992646 14532 webserver.cc:460] Webserver started at http://127.1.192.195:38717/ using document root <none> and password file <none>
I20260809 02:04:17.993245 14532 fs_manager.cc:359] Metadata directory not provided
I20260809 02:04:17.993345 14532 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:04:17.993644 14532 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 02:04:17 dist-test-slave-ss4n krb5kdc[13640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241057, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.195@KRBTEST.COM for HTTP/127.1.192.212@KRBTEST.COM
I20260809 02:04:18.018790 14532 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/minicluster-data/ts-2/data/instance:
uuid: "1938487f07e84cf490c94e5ecef99a14"
format_stamp: "Formatted at 2026-08-09 02:04:18 on dist-test-slave-ss4n"
server_key: "5320a186dc40c91905c22a4cd13592d4"
server_key_iv: "0edd5a700e5a06bff8b45af4c38421f8"
server_key_version: "kuduclusterkey@0"
I20260809 02:04:18.019675 14532 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/minicluster-data/ts-2/wal/instance:
uuid: "1938487f07e84cf490c94e5ecef99a14"
format_stamp: "Formatted at 2026-08-09 02:04:18 on dist-test-slave-ss4n"
server_key: "5320a186dc40c91905c22a4cd13592d4"
server_key_iv: "0edd5a700e5a06bff8b45af4c38421f8"
server_key_version: "kuduclusterkey@0"
I20260809 02:04:18.026381 14532 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260809 02:04:18.051780 14547 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:04:18.053464 14532 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 02:04:18.053651 14532 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/minicluster-data/ts-2/wal
uuid: "1938487f07e84cf490c94e5ecef99a14"
format_stamp: "Formatted at 2026-08-09 02:04:18 on dist-test-slave-ss4n"
server_key: "5320a186dc40c91905c22a4cd13592d4"
server_key_iv: "0edd5a700e5a06bff8b45af4c38421f8"
server_key_version: "kuduclusterkey@0"
I20260809 02:04:18.053789 14532 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:04:18.074906 14532 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:04:18.078740 14532 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:04:18.078939 14532 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:04:18.079646 14532 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:04:18.080704 14532 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:04:18.080771 14532 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:04:18.080824 14532 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:04:18.080904 14532 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:04:18.091805 14532 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.195:35243
I20260809 02:04:18.091885 14665 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.195:35243 every 8 connection(s)
I20260809 02:04:18.093057 14532 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/minicluster-data/ts-2/data/info.pb
I20260809 02:04:18.103108 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 14532
I20260809 02:04:18.103240 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/minicluster-data/ts-2/wal/instance
I20260809 02:04:18.103544 1795 external_mini_cluster.cc:1428] Setting key 790a8bacf66ae3332fe80066fb1fb8fe
Aug 09 02:04:18 dist-test-slave-ss4n krb5kdc[13640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241057, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.195@KRBTEST.COM for kudu/127.1.192.254@KRBTEST.COM
I20260809 02:04:18.111584 14385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:04:18.095197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.195:60013 (local address 127.1.192.254:46009)
0809 02:04:18.095348 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0809 02:04:18.095353 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:04:18.097238 (+ 1885us) server_negotiation.cc:407] Connection header received
0809 02:04:18.098479 (+ 1241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:04:18.098483 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:04:18.098539 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:04:18.098711 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:04:18.100153 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:04:18.100996 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:04:18.102820 (+ 1824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:04:18.103171 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:04:18.103278 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:04:18.107134 (+ 3856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:04:18.107138 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:04:18.107141 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:04:18.107169 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:04:18.109030 (+ 1861us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:04:18.109752 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:04:18.109756 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:04:18.109758 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:04:18.109850 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:04:18.110256 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:04:18.110259 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:04:18.110261 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:04:18.110518 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:04:18.110671 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0809 02:04:18.111265 (+ 594us) server_negotiation.cc:299] Negotiation successful
0809 02:04:18.111399 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 02:04:18.112663 14669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:04:18.095742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:46009 (local address 127.1.192.195:60013)
0809 02:04:18.097045 (+ 1303us) negotiation.cc:107] Waiting for socket to connect
0809 02:04:18.097105 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0809 02:04:18.098220 (+ 1115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:04:18.098910 (+ 690us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:04:18.098919 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:04:18.099293 (+ 374us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:04:18.099961 (+ 668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:04:18.099981 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:04:18.101141 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:04:18.101145 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:04:18.102685 (+ 1540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:04:18.102696 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:04:18.103455 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:04:18.103459 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:04:18.103671 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:04:18.104572 (+ 901us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:04:18.104593 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:04:18.106947 (+ 2354us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:04:18.109183 (+ 2236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:04:18.109191 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:04:18.109210 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:04:18.109654 (+ 444us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:04:18.109952 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:04:18.109956 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:04:18.109958 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:04:18.110147 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:04:18.110639 (+ 492us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:04:18.110646 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:04:18.110965 (+ 319us) client_negotiation.cc:769] Sending connection context
0809 02:04:18.111176 (+ 211us) client_negotiation.cc:240] Negotiation successful
0809 02:04:18.111518 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1019,"thread_start_us":149,"threads_started":1}
I20260809 02:04:18.113945 14666 heartbeater.cc:347] Connected to a master server at 127.1.192.254:46009
I20260809 02:04:18.114216 14666 heartbeater.cc:464] Registering TS with master...
I20260809 02:04:18.114789 14666 heartbeater.cc:511] Master 127.1.192.254:46009 requested a full tablet report, sending...
I20260809 02:04:18.115942 14154 ts_manager.cc:194] Registered new tserver with Master: 1938487f07e84cf490c94e5ecef99a14 (127.1.192.195:35243)
I20260809 02:04:18.116729 14154 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.192.195@KRBTEST.COM'} at 127.1.192.195:60013
I20260809 02:04:18.121843 1795 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 02:04:18.134776 14385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:04:18.124291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56146 (local address 127.1.192.254:46009)
0809 02:04:18.124416 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0809 02:04:18.124421 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:04:18.125030 (+ 609us) server_negotiation.cc:407] Connection header received
0809 02:04:18.125213 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:04:18.125216 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:04:18.125262 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:04:18.125313 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:04:18.126391 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:04:18.127190 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:04:18.128438 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:04:18.128786 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:04:18.128892 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:04:18.130756 (+ 1864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:04:18.130760 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:04:18.130763 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:04:18.130790 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:04:18.132770 (+ 1980us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:04:18.133294 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:04:18.133297 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:04:18.133299 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:04:18.133392 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:04:18.133762 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:04:18.133765 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:04:18.133767 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:04:18.133985 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:04:18.134120 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0809 02:04:18.134469 (+ 349us) server_negotiation.cc:299] Negotiation successful
0809 02:04:18.134606 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 02:04:18.138557 14154 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56146:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 02:04:18.143033 14154 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:04:18.164003 14680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:04:18.155300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.195:35243 (local address 127.0.0.1:55872)
0809 02:04:18.157571 (+ 2271us) negotiation.cc:107] Waiting for socket to connect
0809 02:04:18.157588 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 02:04:18.157687 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:04:18.158719 (+ 1032us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:04:18.158722 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:04:18.158741 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:04:18.158844 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:04:18.158851 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:04:18.160555 (+ 1704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:04:18.160559 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:04:18.162008 (+ 1449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:04:18.162019 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:04:18.163387 (+ 1368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:04:18.163391 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:04:18.163536 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:04:18.163584 (+ 48us) client_negotiation.cc:769] Sending connection context
0809 02:04:18.163704 (+ 120us) client_negotiation.cc:240] Negotiation successful
0809 02:04:18.163834 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2111,"thread_start_us":103,"threads_started":1}
I20260809 02:04:18.164633 14682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:04:18.154757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.194:45489 (local address 127.0.0.1:58060)
0809 02:04:18.157343 (+ 2586us) negotiation.cc:107] Waiting for socket to connect
0809 02:04:18.157397 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 02:04:18.157560 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:04:18.158320 (+ 760us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:04:18.158325 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:04:18.158348 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:04:18.158533 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:04:18.158539 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:04:18.160861 (+ 2322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:04:18.160865 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:04:18.162070 (+ 1205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:04:18.162082 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:04:18.164217 (+ 2135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:04:18.164221 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:04:18.164315 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:04:18.164336 (+ 21us) client_negotiation.cc:769] Sending connection context
0809 02:04:18.164406 (+ 70us) client_negotiation.cc:240] Negotiation successful
0809 02:04:18.164471 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2406,"thread_start_us":119,"threads_started":1}
I20260809 02:04:18.165113 14684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:04:18.155943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58060 (local address 127.1.192.194:45489)
0809 02:04:18.157078 (+ 1135us) server_negotiation.cc:206] Beginning negotiation
0809 02:04:18.157090 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 02:04:18.157708 (+ 618us) server_negotiation.cc:407] Connection header received
0809 02:04:18.157789 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:04:18.157794 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:04:18.158001 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:04:18.158152 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:04:18.158674 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:04:18.160727 (+ 2053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:04:18.163156 (+ 2429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:04:18.163931 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:04:18.164024 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:04:18.164149 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0809 02:04:18.164865 (+ 716us) server_negotiation.cc:299] Negotiation successful
0809 02:04:18.164973 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1027,"thread_start_us":129,"threads_started":1}
I20260809 02:04:18.165120 14685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:04:18.156319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55872 (local address 127.1.192.195:35243)
0809 02:04:18.157954 (+ 1635us) server_negotiation.cc:206] Beginning negotiation
0809 02:04:18.157968 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 02:04:18.157991 (+ 23us) server_negotiation.cc:407] Connection header received
0809 02:04:18.158069 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:04:18.158075 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:04:18.158266 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:04:18.158414 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:04:18.159145 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:04:18.160400 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:04:18.162209 (+ 1809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:04:18.163050 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:04:18.163161 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:04:18.163319 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0809 02:04:18.164869 (+ 1550us) server_negotiation.cc:299] Negotiation successful
0809 02:04:18.164978 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1534,"thread_start_us":137,"threads_started":1}
I20260809 02:04:18.168772 14455 tablet_service.cc:1467] Processing CreateTablet for tablet c4962e3930ef411aba1017e3162f93f3 (DEFAULT_TABLE table=test-table [id=cd3e3c1bec6644a09e0315d868bc2ccb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:04:18.170536 14455 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c4962e3930ef411aba1017e3162f93f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:04:18.173691 14681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:04:18.155535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.193:36437 (local address 127.0.0.1:54354)
0809 02:04:18.158995 (+ 3460us) negotiation.cc:107] Waiting for socket to connect
0809 02:04:18.159009 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 02:04:18.159122 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:04:18.159780 (+ 658us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:04:18.159783 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:04:18.159800 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:04:18.159919 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:04:18.159925 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:04:18.171068 (+ 11143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:04:18.171073 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:04:18.172103 (+ 1030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:04:18.172115 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:04:18.173226 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:04:18.173231 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:04:18.173336 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:04:18.173366 (+ 30us) client_negotiation.cc:769] Sending connection context
0809 02:04:18.173421 (+ 55us) client_negotiation.cc:240] Negotiation successful
0809 02:04:18.173533 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3400,"thread_start_us":99,"threads_started":1}
I20260809 02:04:18.174431 14683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:04:18.155928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54354 (local address 127.1.192.193:36437)
0809 02:04:18.158986 (+ 3058us) server_negotiation.cc:206] Beginning negotiation
0809 02:04:18.158993 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 02:04:18.159224 (+ 231us) server_negotiation.cc:407] Connection header received
0809 02:04:18.159303 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:04:18.159308 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:04:18.159493 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:04:18.159634 (+ 141us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:04:18.169588 (+ 9954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:04:18.170891 (+ 1303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:04:18.172248 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:04:18.173071 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:04:18.173986 (+ 915us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:04:18.174142 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0809 02:04:18.174198 (+ 56us) server_negotiation.cc:299] Negotiation successful
0809 02:04:18.174293 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2673,"thread_start_us":89,"threads_started":1}
I20260809 02:04:18.177435 14301 tablet_service.cc:1467] Processing CreateTablet for tablet c4962e3930ef411aba1017e3162f93f3 (DEFAULT_TABLE table=test-table [id=cd3e3c1bec6644a09e0315d868bc2ccb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:04:18.178649 14301 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c4962e3930ef411aba1017e3162f93f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:04:18.186281 14686 tablet_bootstrap.cc:492] T c4962e3930ef411aba1017e3162f93f3 P 23a14f53e268416f816e5a1f62b3605d: Bootstrap starting.
I20260809 02:04:18.188690 14686 tablet_bootstrap.cc:654] T c4962e3930ef411aba1017e3162f93f3 P 23a14f53e268416f816e5a1f62b3605d: Neither blocks nor log segments found. Creating new log.
I20260809 02:04:18.189603 14686 log.cc:824] T c4962e3930ef411aba1017e3162f93f3 P 23a14f53e268416f816e5a1f62b3605d: Log is configured to *not* fsync() on all Append() calls
I20260809 02:04:18.192339 14589 tablet_service.cc:1467] Processing CreateTablet for tablet c4962e3930ef411aba1017e3162f93f3 (DEFAULT_TABLE table=test-table [id=cd3e3c1bec6644a09e0315d868bc2ccb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:04:18.193542 14589 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c4962e3930ef411aba1017e3162f93f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:04:18.197825 14687 tablet_bootstrap.cc:492] T c4962e3930ef411aba1017e3162f93f3 P 0e0a5584605d40f39f6161b2dc18ec1f: Bootstrap starting.
I20260809 02:04:18.200398 14687 tablet_bootstrap.cc:654] T c4962e3930ef411aba1017e3162f93f3 P 0e0a5584605d40f39f6161b2dc18ec1f: Neither blocks nor log segments found. Creating new log.
I20260809 02:04:18.202075 14687 log.cc:824] T c4962e3930ef411aba1017e3162f93f3 P 0e0a5584605d40f39f6161b2dc18ec1f: Log is configured to *not* fsync() on all Append() calls
I20260809 02:04:18.205902 14686 tablet_bootstrap.cc:492] T c4962e3930ef411aba1017e3162f93f3 P 23a14f53e268416f816e5a1f62b3605d: No bootstrap required, opened a new log
I20260809 02:04:18.206140 14686 ts_tablet_manager.cc:1397] T c4962e3930ef411aba1017e3162f93f3 P 23a14f53e268416f816e5a1f62b3605d: Time spent bootstrapping tablet: real 0.020s user 0.005s sys 0.000s
I20260809 02:04:18.210942 14686 raft_consensus.cc:348] T c4962e3930ef411aba1017e3162f93f3 P 23a14f53e268416f816e5a1f62b3605d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0a5584605d40f39f6161b2dc18ec1f" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 45489 } } peers { permanent_uuid: "23a14f53e268416f816e5a1f62b3605d" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 36437 } } peers { permanent_uuid: "1938487f07e84cf490c94e5ecef99a14" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 35243 } }
I20260809 02:04:18.211269 14686 raft_consensus.cc:374] T c4962e3930ef411aba1017e3162f93f3 P 23a14f53e268416f816e5a1f62b3605d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:04:18.211330 14686 raft_consensus.cc:729] T c4962e3930ef411aba1017e3162f93f3 P 23a14f53e268416f816e5a1f62b3605d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23a14f53e268416f816e5a1f62b3605d, State: Initialized, Role: FOLLOWER
I20260809 02:04:18.211845 14686 consensus_queue.cc:260] T c4962e3930ef411aba1017e3162f93f3 P 23a14f53e268416f816e5a1f62b3605d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0a5584605d40f39f6161b2dc18ec1f" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 45489 } } peers { permanent_uuid: "23a14f53e268416f816e5a1f62b3605d" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 36437 } } peers { permanent_uuid: "1938487f07e84cf490c94e5ecef99a14" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 35243 } }
I20260809 02:04:18.212971 14686 ts_tablet_manager.cc:1428] T c4962e3930ef411aba1017e3162f93f3 P 23a14f53e268416f816e5a1f62b3605d: Time spent starting tablet: real 0.007s user 0.003s sys 0.004s
I20260809 02:04:18.213346 14383 heartbeater.cc:503] Master 127.1.192.254:46009 was elected leader, sending a full tablet report...
I20260809 02:04:18.219509 14687 tablet_bootstrap.cc:492] T c4962e3930ef411aba1017e3162f93f3 P 0e0a5584605d40f39f6161b2dc18ec1f: No bootstrap required, opened a new log
I20260809 02:04:18.219756 14687 ts_tablet_manager.cc:1397] T c4962e3930ef411aba1017e3162f93f3 P 0e0a5584605d40f39f6161b2dc18ec1f: Time spent bootstrapping tablet: real 0.022s user 0.005s sys 0.000s
I20260809 02:04:18.220144 14691 tablet_bootstrap.cc:492] T c4962e3930ef411aba1017e3162f93f3 P 1938487f07e84cf490c94e5ecef99a14: Bootstrap starting.
I20260809 02:04:18.224956 14691 tablet_bootstrap.cc:654] T c4962e3930ef411aba1017e3162f93f3 P 1938487f07e84cf490c94e5ecef99a14: Neither blocks nor log segments found. Creating new log.
I20260809 02:04:18.224871 14687 raft_consensus.cc:348] T c4962e3930ef411aba1017e3162f93f3 P 0e0a5584605d40f39f6161b2dc18ec1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0a5584605d40f39f6161b2dc18ec1f" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 45489 } } peers { permanent_uuid: "23a14f53e268416f816e5a1f62b3605d" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 36437 } } peers { permanent_uuid: "1938487f07e84cf490c94e5ecef99a14" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 35243 } }
I20260809 02:04:18.225178 14687 raft_consensus.cc:374] T c4962e3930ef411aba1017e3162f93f3 P 0e0a5584605d40f39f6161b2dc18ec1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:04:18.225258 14687 raft_consensus.cc:729] T c4962e3930ef411aba1017e3162f93f3 P 0e0a5584605d40f39f6161b2dc18ec1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e0a5584605d40f39f6161b2dc18ec1f, State: Initialized, Role: FOLLOWER
I20260809 02:04:18.225817 14687 consensus_queue.cc:260] T c4962e3930ef411aba1017e3162f93f3 P 0e0a5584605d40f39f6161b2dc18ec1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0a5584605d40f39f6161b2dc18ec1f" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 45489 } } peers { permanent_uuid: "23a14f53e268416f816e5a1f62b3605d" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 36437 } } peers { permanent_uuid: "1938487f07e84cf490c94e5ecef99a14" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 35243 } }
I20260809 02:04:18.226284 14691 log.cc:824] T c4962e3930ef411aba1017e3162f93f3 P 1938487f07e84cf490c94e5ecef99a14: Log is configured to *not* fsync() on all Append() calls
I20260809 02:04:18.226933 14687 ts_tablet_manager.cc:1428] T c4962e3930ef411aba1017e3162f93f3 P 0e0a5584605d40f39f6161b2dc18ec1f: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260809 02:04:18.227298 14527 heartbeater.cc:503] Master 127.1.192.254:46009 was elected leader, sending a full tablet report...
I20260809 02:04:18.229036 14691 tablet_bootstrap.cc:492] T c4962e3930ef411aba1017e3162f93f3 P 1938487f07e84cf490c94e5ecef99a14: No bootstrap required, opened a new log
I20260809 02:04:18.229282 14691 ts_tablet_manager.cc:1397] T c4962e3930ef411aba1017e3162f93f3 P 1938487f07e84cf490c94e5ecef99a14: Time spent bootstrapping tablet: real 0.015s user 0.000s sys 0.005s
Aug 09 02:04:18 dist-test-slave-ss4n krb5kdc[13640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241057, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.254@KRBTEST.COM for HTTP/127.1.192.212@KRBTEST.COM
I20260809 02:04:18.233549 14691 raft_consensus.cc:348] T c4962e3930ef411aba1017e3162f93f3 P 1938487f07e84cf490c94e5ecef99a14 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0a5584605d40f39f6161b2dc18ec1f" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 45489 } } peers { permanent_uuid: "23a14f53e268416f816e5a1f62b3605d" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 36437 } } peers { permanent_uuid: "1938487f07e84cf490c94e5ecef99a14" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 35243 } }
W20260809 02:04:18.233844 14528 tablet.cc:2369] T c4962e3930ef411aba1017e3162f93f3 P 0e0a5584605d40f39f6161b2dc18ec1f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 02:04:18.233867 14691 raft_consensus.cc:374] T c4962e3930ef411aba1017e3162f93f3 P 1938487f07e84cf490c94e5ecef99a14 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:04:18.233933 14691 raft_consensus.cc:729] T c4962e3930ef411aba1017e3162f93f3 P 1938487f07e84cf490c94e5ecef99a14 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1938487f07e84cf490c94e5ecef99a14, State: Initialized, Role: FOLLOWER
I20260809 02:04:18.234462 14691 consensus_queue.cc:260] T c4962e3930ef411aba1017e3162f93f3 P 1938487f07e84cf490c94e5ecef99a14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0a5584605d40f39f6161b2dc18ec1f" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 45489 } } peers { permanent_uuid: "23a14f53e268416f816e5a1f62b3605d" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 36437 } } peers { permanent_uuid: "1938487f07e84cf490c94e5ecef99a14" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 35243 } }
I20260809 02:04:18.235728 14666 heartbeater.cc:503] Master 127.1.192.254:46009 was elected leader, sending a full tablet report...
I20260809 02:04:18.237813 14691 ts_tablet_manager.cc:1428] T c4962e3930ef411aba1017e3162f93f3 P 1938487f07e84cf490c94e5ecef99a14: Time spent starting tablet: real 0.008s user 0.006s sys 0.001s
I20260809 02:04:18.279927 14697 raft_consensus.cc:482] T c4962e3930ef411aba1017e3162f93f3 P 1938487f07e84cf490c94e5ecef99a14 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:04:18.280160 14697 raft_consensus.cc:504] T c4962e3930ef411aba1017e3162f93f3 P 1938487f07e84cf490c94e5ecef99a14 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0a5584605d40f39f6161b2dc18ec1f" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 45489 } } peers { permanent_uuid: "23a14f53e268416f816e5a1f62b3605d" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 36437 } } peers { permanent_uuid: "1938487f07e84cf490c94e5ecef99a14" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 35243 } }
I20260809 02:04:18.281548 14697 leader_election.cc:288] T c4962e3930ef411aba1017e3162f93f3 P 1938487f07e84cf490c94e5ecef99a14 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e0a5584605d40f39f6161b2dc18ec1f (127.1.192.194:45489), 23a14f53e268416f816e5a1f62b3605d (127.1.192.193:36437)
I20260809 02:04:18.285738 14669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:04:18.281784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.194:45489 (local address 127.1.192.195:34229)
0809 02:04:18.281988 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0809 02:04:18.282004 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 02:04:18.282099 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:04:18.282534 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:04:18.282539 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:04:18.282567 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:04:18.282709 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:04:18.282717 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:04:18.283570 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:04:18.283575 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:04:18.284641 (+ 1066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:04:18.284659 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:04:18.285330 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:04:18.285334 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:04:18.285430 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:04:18.285452 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 02:04:18.285554 (+ 102us) client_negotiation.cc:240] Negotiation successful
0809 02:04:18.285626 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"mutex_wait_us":30}
I20260809 02:04:18.286265 14684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:04:18.281839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.195:34229 (local address 127.1.192.194:45489)
0809 02:04:18.282291 (+ 452us) server_negotiation.cc:206] Beginning negotiation
0809 02:04:18.282296 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:04:18.282309 (+ 13us) server_negotiation.cc:407] Connection header received
0809 02:04:18.282363 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:04:18.282366 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:04:18.282412 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:04:18.282498 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:04:18.282828 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:04:18.283457 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:04:18.284804 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:04:18.285205 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:04:18.285338 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:04:18.285394 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 02:04:18.286106 (+ 712us) server_negotiation.cc:299] Negotiation successful
0809 02:04:18.286176 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362}
I20260809 02:04:18.286403 14683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:04:18.282189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.195:57111 (local address 127.1.192.193:36437)
0809 02:04:18.282320 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0809 02:04:18.282324 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:04:18.282634 (+ 310us) server_negotiation.cc:407] Connection header received
0809 02:04:18.282739 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:04:18.282743 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:04:18.282798 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:04:18.282877 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:04:18.283171 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:04:18.283916 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:04:18.285161 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:04:18.285670 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:04:18.285789 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:04:18.285839 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 02:04:18.286183 (+ 344us) server_negotiation.cc:299] Negotiation successful
0809 02:04:18.286271 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 02:04:18.286816 14699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:04:18.282062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.193:36437 (local address 127.1.192.195:57111)
0809 02:04:18.282533 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0809 02:04:18.282545 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:04:18.282648 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:04:18.282887 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:04:18.282890 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:04:18.282907 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:04:18.283015 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:04:18.283021 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:04:18.284013 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:04:18.284018 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:04:18.285013 (+ 995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:04:18.285023 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:04:18.285931 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:04:18.285935 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:04:18.286033 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:04:18.286054 (+ 21us) client_negotiation.cc:769] Sending connection context
0809 02:04:18.286548 (+ 494us) client_negotiation.cc:240] Negotiation successful
0809 02:04:18.286642 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":386,"mutex_wait_us":1,"thread_start_us":157,"threads_started":1}
I20260809 02:04:18.286899 14477 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4962e3930ef411aba1017e3162f93f3" candidate_uuid: "1938487f07e84cf490c94e5ecef99a14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e0a5584605d40f39f6161b2dc18ec1f" is_pre_election: true
I20260809 02:04:18.287251 14477 raft_consensus.cc:2454] T c4962e3930ef411aba1017e3162f93f3 P 0e0a5584605d40f39f6161b2dc18ec1f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1938487f07e84cf490c94e5ecef99a14 in term 0.
I20260809 02:04:18.287504 14315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4962e3930ef411aba1017e3162f93f3" candidate_uuid: "1938487f07e84cf490c94e5ecef99a14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23a14f53e268416f816e5a1f62b3605d" is_pre_election: true
I20260809 02:04:18.287783 14315 raft_consensus.cc:2454] T c4962e3930ef411aba1017e3162f93f3 P 23a14f53e268416f816e5a1f62b3605d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1938487f07e84cf490c94e5ecef99a14 in term 0.
I20260809 02:04:18.287789 14550 leader_election.cc:302] T c4962e3930ef411aba1017e3162f93f3 P 1938487f07e84cf490c94e5ecef99a14 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e0a5584605d40f39f6161b2dc18ec1f, 1938487f07e84cf490c94e5ecef99a14; no voters:
I20260809 02:04:18.288117 14697 raft_consensus.cc:2781] T c4962e3930ef411aba1017e3162f93f3 P 1938487f07e84cf490c94e5ecef99a14 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:04:18.288192 14697 raft_consensus.cc:482] T c4962e3930ef411aba1017e3162f93f3 P 1938487f07e84cf490c94e5ecef99a14 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:04:18.288228 14697 raft_consensus.cc:3037] T c4962e3930ef411aba1017e3162f93f3 P 1938487f07e84cf490c94e5ecef99a14 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:04:18.289371 14697 raft_consensus.cc:504] T c4962e3930ef411aba1017e3162f93f3 P 1938487f07e84cf490c94e5ecef99a14 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0a5584605d40f39f6161b2dc18ec1f" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 45489 } } peers { permanent_uuid: "23a14f53e268416f816e5a1f62b3605d" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 36437 } } peers { permanent_uuid: "1938487f07e84cf490c94e5ecef99a14" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 35243 } }
I20260809 02:04:18.289759 14697 leader_election.cc:288] T c4962e3930ef411aba1017e3162f93f3 P 1938487f07e84cf490c94e5ecef99a14 [CANDIDATE]: Term 1 election: Requested vote from peers 0e0a5584605d40f39f6161b2dc18ec1f (127.1.192.194:45489), 23a14f53e268416f816e5a1f62b3605d (127.1.192.193:36437)
I20260809 02:04:18.290195 14315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4962e3930ef411aba1017e3162f93f3" candidate_uuid: "1938487f07e84cf490c94e5ecef99a14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23a14f53e268416f816e5a1f62b3605d"
I20260809 02:04:18.290320 14315 raft_consensus.cc:3037] T c4962e3930ef411aba1017e3162f93f3 P 23a14f53e268416f816e5a1f62b3605d [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:04:18.290562 14477 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4962e3930ef411aba1017e3162f93f3" candidate_uuid: "1938487f07e84cf490c94e5ecef99a14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e0a5584605d40f39f6161b2dc18ec1f"
I20260809 02:04:18.290720 14477 raft_consensus.cc:3037] T c4962e3930ef411aba1017e3162f93f3 P 0e0a5584605d40f39f6161b2dc18ec1f [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:04:18.291731 14477 raft_consensus.cc:2454] T c4962e3930ef411aba1017e3162f93f3 P 0e0a5584605d40f39f6161b2dc18ec1f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1938487f07e84cf490c94e5ecef99a14 in term 1.
I20260809 02:04:18.291734 14315 raft_consensus.cc:2454] T c4962e3930ef411aba1017e3162f93f3 P 23a14f53e268416f816e5a1f62b3605d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1938487f07e84cf490c94e5ecef99a14 in term 1.
I20260809 02:04:18.292078 14550 leader_election.cc:302] T c4962e3930ef411aba1017e3162f93f3 P 1938487f07e84cf490c94e5ecef99a14 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1938487f07e84cf490c94e5ecef99a14, 23a14f53e268416f816e5a1f62b3605d; no voters:
I20260809 02:04:18.292359 14697 raft_consensus.cc:2781] T c4962e3930ef411aba1017e3162f93f3 P 1938487f07e84cf490c94e5ecef99a14 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:04:18.292605 14697 raft_consensus.cc:686] T c4962e3930ef411aba1017e3162f93f3 P 1938487f07e84cf490c94e5ecef99a14 [term 1 LEADER]: Becoming Leader. State: Replica: 1938487f07e84cf490c94e5ecef99a14, State: Running, Role: LEADER
I20260809 02:04:18.292964 14697 consensus_queue.cc:237] T c4962e3930ef411aba1017e3162f93f3 P 1938487f07e84cf490c94e5ecef99a14 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0a5584605d40f39f6161b2dc18ec1f" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 45489 } } peers { permanent_uuid: "23a14f53e268416f816e5a1f62b3605d" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 36437 } } peers { permanent_uuid: "1938487f07e84cf490c94e5ecef99a14" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 35243 } }
I20260809 02:04:18.296743 14154 catalog_manager.cc:5495] T c4962e3930ef411aba1017e3162f93f3 P 1938487f07e84cf490c94e5ecef99a14 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1938487f07e84cf490c94e5ecef99a14 (127.1.192.195). New cstate: current_term: 1 leader_uuid: "1938487f07e84cf490c94e5ecef99a14" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e0a5584605d40f39f6161b2dc18ec1f" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 45489 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "23a14f53e268416f816e5a1f62b3605d" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 36437 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1938487f07e84cf490c94e5ecef99a14" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 35243 } health_report { overall_health: HEALTHY } } }
I20260809 02:04:18.318349 14685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:04:18.313888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55886 (local address 127.1.192.195:35243)
0809 02:04:18.314042 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0809 02:04:18.314047 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:04:18.314235 (+ 188us) server_negotiation.cc:407] Connection header received
0809 02:04:18.314340 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:04:18.314344 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:04:18.314416 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:04:18.314546 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:04:18.314820 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:04:18.315549 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:04:18.316503 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:04:18.316853 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:04:18.316970 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:04:18.317322 (+ 352us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:04:18.317936 (+ 614us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:04:18.318065 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0809 02:04:18.318131 (+ 66us) server_negotiation.cc:299] Negotiation successful
0809 02:04:18.318197 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 02:04:18.330714 14477 raft_consensus.cc:1261] T c4962e3930ef411aba1017e3162f93f3 P 0e0a5584605d40f39f6161b2dc18ec1f [term 1 FOLLOWER]: Refusing update from remote peer 1938487f07e84cf490c94e5ecef99a14: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:04:18.330704 14315 raft_consensus.cc:1261] T c4962e3930ef411aba1017e3162f93f3 P 23a14f53e268416f816e5a1f62b3605d [term 1 FOLLOWER]: Refusing update from remote peer 1938487f07e84cf490c94e5ecef99a14: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:04:18.331604 14697 consensus_queue.cc:1035] T c4962e3930ef411aba1017e3162f93f3 P 1938487f07e84cf490c94e5ecef99a14 [LEADER]: Connected to new peer: Peer: permanent_uuid: "23a14f53e268416f816e5a1f62b3605d" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 36437 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:04:18.331898 14700 consensus_queue.cc:1035] T c4962e3930ef411aba1017e3162f93f3 P 1938487f07e84cf490c94e5ecef99a14 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e0a5584605d40f39f6161b2dc18ec1f" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 45489 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:04:18.339092 14703 mvcc.cc:205] Tried to move back new op lower bound from 7316443374894039040 to 7316443374770987008. Current Snapshot: MvccSnapshot[applied={T|T < 7316443374894039040}]
I20260809 02:04:18.343662 14704 mvcc.cc:205] Tried to move back new op lower bound from 7316443374894039040 to 7316443374770987008. Current Snapshot: MvccSnapshot[applied={T|T < 7316443374894039040}]
I20260809 02:04:18.351361 14707 mvcc.cc:205] Tried to move back new op lower bound from 7316443374894039040 to 7316443374770987008. Current Snapshot: MvccSnapshot[applied={T|T < 7316443374894039040}]
I20260809 02:04:18.356498 14153 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56146:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:04:18.356709 14153 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56146:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:04:18.359869 14153 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f4210c06d2744ebc88b4288e24f62180: Sending DeleteTablet for 3 replicas of tablet c4962e3930ef411aba1017e3162f93f3
I20260809 02:04:18.360879 14455 tablet_service.cc:1514] Processing DeleteTablet for tablet c4962e3930ef411aba1017e3162f93f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:04:18 UTC) from {username='kudu', principal='kudu/127.1.192.254@KRBTEST.COM'} at 127.0.0.1:58060
I20260809 02:04:18.362066 14301 tablet_service.cc:1514] Processing DeleteTablet for tablet c4962e3930ef411aba1017e3162f93f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:04:18 UTC) from {username='kudu', principal='kudu/127.1.192.254@KRBTEST.COM'} at 127.0.0.1:54354
I20260809 02:04:18.363081 14589 tablet_service.cc:1514] Processing DeleteTablet for tablet c4962e3930ef411aba1017e3162f93f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:04:18 UTC) from {username='kudu', principal='kudu/127.1.192.254@KRBTEST.COM'} at 127.0.0.1:55872
I20260809 02:04:18.364698 14715 tablet_replica.cc:330] T c4962e3930ef411aba1017e3162f93f3 P 1938487f07e84cf490c94e5ecef99a14: stopping tablet replica
I20260809 02:04:18.365051 14715 raft_consensus.cc:2229] T c4962e3930ef411aba1017e3162f93f3 P 1938487f07e84cf490c94e5ecef99a14 [term 1 LEADER]: Raft consensus shutting down.
I20260809 02:04:18.365434 14715 raft_consensus.cc:2258] T c4962e3930ef411aba1017e3162f93f3 P 1938487f07e84cf490c94e5ecef99a14 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:04:18.365818 14714 tablet_replica.cc:330] T c4962e3930ef411aba1017e3162f93f3 P 23a14f53e268416f816e5a1f62b3605d: stopping tablet replica
I20260809 02:04:18.366111 14714 raft_consensus.cc:2229] T c4962e3930ef411aba1017e3162f93f3 P 23a14f53e268416f816e5a1f62b3605d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:04:18.366415 14714 raft_consensus.cc:2258] T c4962e3930ef411aba1017e3162f93f3 P 23a14f53e268416f816e5a1f62b3605d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:04:18.366801 14713 tablet_replica.cc:330] T c4962e3930ef411aba1017e3162f93f3 P 0e0a5584605d40f39f6161b2dc18ec1f: stopping tablet replica
I20260809 02:04:18.367090 14713 raft_consensus.cc:2229] T c4962e3930ef411aba1017e3162f93f3 P 0e0a5584605d40f39f6161b2dc18ec1f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:04:18.367357 14713 raft_consensus.cc:2258] T c4962e3930ef411aba1017e3162f93f3 P 0e0a5584605d40f39f6161b2dc18ec1f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:04:18.368530 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 14241
I20260809 02:04:18.368714 14713 ts_tablet_manager.cc:1905] T c4962e3930ef411aba1017e3162f93f3 P 0e0a5584605d40f39f6161b2dc18ec1f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:04:18.372709 14713 ts_tablet_manager.cc:1918] T c4962e3930ef411aba1017e3162f93f3 P 0e0a5584605d40f39f6161b2dc18ec1f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:04:18.372833 14713 log.cc:1196] T c4962e3930ef411aba1017e3162f93f3 P 0e0a5584605d40f39f6161b2dc18ec1f: Deleting WAL directory at /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/minicluster-data/ts-1/wal/wals/c4962e3930ef411aba1017e3162f93f3
I20260809 02:04:18.373176 14713 ts_tablet_manager.cc:1939] T c4962e3930ef411aba1017e3162f93f3 P 0e0a5584605d40f39f6161b2dc18ec1f: Deleting consensus metadata
I20260809 02:04:18.374914 14138 catalog_manager.cc:4841] TS 0e0a5584605d40f39f6161b2dc18ec1f (127.1.192.194:45489): tablet c4962e3930ef411aba1017e3162f93f3 (table test-table [id=cd3e3c1bec6644a09e0315d868bc2ccb]) successfully deleted
I20260809 02:04:18.375351 14715 ts_tablet_manager.cc:1905] T c4962e3930ef411aba1017e3162f93f3 P 1938487f07e84cf490c94e5ecef99a14: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:04:18.378094 14715 ts_tablet_manager.cc:1918] T c4962e3930ef411aba1017e3162f93f3 P 1938487f07e84cf490c94e5ecef99a14: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:04:18.378209 14715 log.cc:1196] T c4962e3930ef411aba1017e3162f93f3 P 1938487f07e84cf490c94e5ecef99a14: Deleting WAL directory at /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786240941968412-1795-0/minicluster-data/ts-2/wal/wals/c4962e3930ef411aba1017e3162f93f3
I20260809 02:04:18.378542 14715 ts_tablet_manager.cc:1939] T c4962e3930ef411aba1017e3162f93f3 P 1938487f07e84cf490c94e5ecef99a14: Deleting consensus metadata
I20260809 02:04:18.379882 14137 catalog_manager.cc:4841] TS 1938487f07e84cf490c94e5ecef99a14 (127.1.192.195:35243): tablet c4962e3930ef411aba1017e3162f93f3 (table test-table [id=cd3e3c1bec6644a09e0315d868bc2ccb]) successfully deleted
W20260809 02:04:18.381641 14138 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.192.193:36437 (error 108)
I20260809 02:04:18.382547 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 14389
I20260809 02:04:18.391970 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 14532
I20260809 02:04:18.394011 14681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:04:18.393573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.193:36437 (local address 127.0.0.1:54364)
0809 02:04:18.393807 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0809 02:04:18.393892 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.193:36437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133}
W20260809 02:04:18.394373 14138 catalog_manager.cc:4584] TS 23a14f53e268416f816e5a1f62b3605d (127.1.192.193:36437): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c4962e3930ef411aba1017e3162f93f3: Network error: Client connection negotiation failed: client connection to 127.1.192.193:36437: connect: Connection refused (error 111)
I20260809 02:04:18.400394 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 14120
I20260809 02:04:18.408685 1795 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260809 02:04:18.754739 1795 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260809 02:04:18.754920 1795 mini_ranger.cc:66] Stopping Ranger...
I20260809 02:04:19.100783 1795 mini_ranger.cc:68] Stopped Ranger
2026-08-09 02:04:19.101 UTC [13674] LOG: received smart shutdown request
2026-08-09 02:04:19.105 UTC [13674] LOG: background worker "logical replication launcher" (PID 13683) exited with exit code 1
2026-08-09 02:04:19.105 UTC [13678] LOG: shutting down
2026-08-09 02:04:19.308 UTC [13674] LOG: database system is shut down
2026-08-09T02:04:19Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (51475 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:04:19 dist-test-slave-ss4n krb5kdc[14725](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:04:19 dist-test-slave-ss4n krb5kdc[14725](info): set up 2 sockets
Aug 09 02:04:19 dist-test-slave-ss4n krb5kdc[14725](info): commencing operation
krb5kdc: starting...
W20260809 02:04:20.612452 1795 mini_kdc.cc:121] Time spent starting KDC: real 1.155s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/krb5kdc/test-user.keytab.
Aug 09 02:04:20 dist-test-slave-ss4n krb5kdc[14725](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241060, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:04:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:04:20Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.1.192.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.1.192.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.1.192.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/krb5kdc/rangeradmin_127.1.192.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.1.192.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/krb5kdc/rangeradmin_127.1.192.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.1.192.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.1.192.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.1.192.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/krb5kdc/rangerlookup_127.1.192.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.1.192.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/krb5kdc/rangerlookup_127.1.192.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.1.192.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.1.192.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.1.192.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/krb5kdc/HTTP_127.1.192.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.1.192.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/krb5kdc/HTTP_127.1.192.212@KRBTEST.COM.keytab.
I20260809 02:04:20.818506 1795 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskRWTcX9/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/postgres -l logfile start
2026-08-09 02:04:23.681 UTC [14756] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-09 02:04:23.681 UTC [14756] LOG: listening on IPv4 address "127.1.192.212", port 45581
2026-08-09 02:04:23.685 UTC [14756] LOG: listening on Unix socket "/tmp/.s.PGSQL.45581"
2026-08-09 02:04:23.708 UTC [14759] LOG: database system was shut down at 2026-08-09 02:04:21 UTC
2026-08-09 02:04:23.713 UTC [14756] LOG: database system is ready to accept connections
I20260809 02:04:24.791793 1795 mini_postgres.cc:96] Postgres bound to 45581
2026-08-09 02:04:24.799 UTC [14767] FATAL: database "slave" does not exist
127.1.192.212:45581 - accepting connections
I20260809 02:04:24.799662 1795 mini_ranger.cc:161] Starting Ranger...
I20260809 02:04:24.816805 1795 mini_ranger.cc:84] Created miniranger Postgres user
I20260809 02:04:25.413137 1795 mini_ranger.cc:87] Created ranger Postgres database
I20260809 02:04:25.413244 1795 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/ranger-admin
2026-08-09 02:04:25,719 [I] DB FLAVOR :POSTGRES
2026-08-09 02:04:25,719 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 02:04:25,719 [I] Checking connection..
2026-08-09 02:04:25,719 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRWTcX9/build/debug/bin/postgresql.jar:/tmp/dist-test-taskRWTcX9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.192.212:45581/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-09 02:04:25,997 [I] Checking connection passed.
2026-08-09 02:04:25,998 [I] --------- Verifying version history table ---------
2026-08-09 02:04:25,998 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRWTcX9/build/debug/bin/postgresql.jar:/tmp/dist-test-taskRWTcX9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.192.212:45581/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 02:04:26,268 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 02:04:26,268 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRWTcX9/build/debug/bin/postgresql.jar:/tmp/dist-test-taskRWTcX9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.192.212:45581/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 02:04:26,544 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 02:04:26,544 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-09 02:04:26,544 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRWTcX9/build/debug/bin/postgresql.jar:/tmp/dist-test-taskRWTcX9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.192.212:45581/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskRWTcX9/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-09 02:04:26.821 UTC [14836] WARNING: there is no transaction in progress
2026-08-09 02:04:26,829 [I] create_dbversion_catalog.sql file imported successfully
2026-08-09 02:04:26,830 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRWTcX9/build/debug/bin/postgresql.jar:/tmp/dist-test-taskRWTcX9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.192.212:45581/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 02:04:27,096 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-09 02:04:27,096 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 02:04:27,097 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRWTcX9/build/debug/bin/postgresql.jar:/tmp/dist-test-taskRWTcX9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.192.212:45581/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-09 02:04:27,359 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRWTcX9/build/debug/bin/postgresql.jar:/tmp/dist-test-taskRWTcX9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.192.212:45581/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-09 02:04:27,625 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRWTcX9/build/debug/bin/postgresql.jar:/tmp/dist-test-taskRWTcX9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.192.212:45581/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-ss4n.c.gcp-upstream.internal','N') ;"
2026-08-09 02:04:27,898 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRWTcX9/build/debug/bin/postgresql.jar:/tmp/dist-test-taskRWTcX9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.192.212:45581/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 02:04:28,180 [I] Table x_portal_user does not exist in database ranger
2026-08-09 02:04:28,181 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRWTcX9/build/debug/bin/postgresql.jar:/tmp/dist-test-taskRWTcX9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.192.212:45581/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 02:04:28,460 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-09 02:04:28,460 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRWTcX9/build/debug/bin/postgresql.jar:/tmp/dist-test-taskRWTcX9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.192.212:45581/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 02:04:28,751 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRWTcX9/build/debug/bin/postgresql.jar:/tmp/dist-test-taskRWTcX9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.192.212:45581/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 02:04:29,036 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-09 02:04:29,036 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRWTcX9/build/debug/bin/postgresql.jar:/tmp/dist-test-taskRWTcX9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.192.212:45581/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskRWTcX9/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-09 02:04:30.141 UTC [14951] WARNING: there is no transaction in progress
2026-08-09 02:04:30.160 UTC [14951] WARNING: there is no transaction in progress
2026-08-09 02:04:30.178 UTC [14951] WARNING: there is no transaction in progress
2026-08-09 02:04:30.195 UTC [14951] WARNING: there is no transaction in progress
2026-08-09 02:04:30.214 UTC [14951] WARNING: there is no transaction in progress
2026-08-09 02:04:30.236 UTC [14951] WARNING: there is no transaction in progress
2026-08-09 02:04:30.306 UTC [14951] WARNING: there is no transaction in progress
2026-08-09 02:04:30.314 UTC [14951] WARNING: there is no transaction in progress
2026-08-09 02:04:30.332 UTC [14951] WARNING: there is no transaction in progress
2026-08-09 02:04:30.345 UTC [14951] WARNING: there is no transaction in progress
2026-08-09 02:04:30.357 UTC [14951] WARNING: there is no transaction in progress
2026-08-09 02:04:30.373 UTC [14951] WARNING: there is no transaction in progress
2026-08-09 02:04:30.385 UTC [14951] WARNING: there is no transaction in progress
2026-08-09 02:04:30.392 UTC [14951] WARNING: there is no transaction in progress
2026-08-09 02:04:30.400 UTC [14951] WARNING: there is no transaction in progress
2026-08-09 02:04:30.407 UTC [14951] WARNING: there is no transaction in progress
2026-08-09 02:04:31,138 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-09 02:04:31,138 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRWTcX9/build/debug/bin/postgresql.jar:/tmp/dist-test-taskRWTcX9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.192.212:45581/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-09 02:04:31,397 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-09 02:04:31,398 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRWTcX9/build/debug/bin/postgresql.jar:/tmp/dist-test-taskRWTcX9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.192.212:45581/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 02:04:31,663 [I] Table x_portal_user already exists in database 'ranger'
2026-08-09 02:04:31,663 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRWTcX9/build/debug/bin/postgresql.jar:/tmp/dist-test-taskRWTcX9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.192.212:45581/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 02:04:31,950 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-09 02:04:31,951 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRWTcX9/build/debug/bin/postgresql.jar:/tmp/dist-test-taskRWTcX9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.192.212:45581/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 02:04:32,225 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRWTcX9/build/debug/bin/postgresql.jar:/tmp/dist-test-taskRWTcX9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.192.212:45581/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 02:04:32,500 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRWTcX9/build/debug/bin/postgresql.jar:/tmp/dist-test-taskRWTcX9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.192.212:45581/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-ss4n.c.gcp-upstream.internal';"
2026-08-09 02:04:32,754 [I] CORE_DB_SCHEMA import status has been updated
2026-08-09 02:04:32,754 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRWTcX9/build/debug/bin/postgresql.jar:/tmp/dist-test-taskRWTcX9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.192.212:45581/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-09 02:04:33,022 [I] DB_PATCHES have already been applied
I20260809 02:04:33.028008 1795 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/ranger-admin:/tmp/dist-test-taskRWTcX9/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskRWTcX9/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskRWTcX9/build/debug/bin/postgresql.jar:/tmp/dist-test-taskRWTcX9/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260809 02:04:33.028126 1795 mini_ranger.cc:193] Using host: 127.1.192.212
I20260809 02:04:33.032719 1795 mini_ranger.cc:227] Ranger admin URL: http://127.1.192.212:35835
Aug 09, 2026 2:04:33 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 2:04:33 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 2:04:33 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/ranger-admin/ews/webapp
Aug 09, 2026 2:04:33 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 09, 2026 2:04:33 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 09, 2026 2:04:33 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/ranger-admin/ews/webapp].
Aug 09, 2026 2:04:33 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260809 02:04:34.265431 1795 mini_ranger.cc:160] Time spent starting Ranger: real 9.466s user 0.000s sys 0.013s
Aug 09, 2026 2:04:55 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 09, 2026 2:04:56 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 09, 2026 2:04:56 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 09, 2026 2:04:56 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 09, 2026 2:04:56 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 09, 2026 2:04:56 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 09, 2026 2:04:56 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 09, 2026 2:04:56 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 09, 2026 2:04:56 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 09, 2026 2:04:56 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 09, 2026 2:04:56 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 09, 2026 2:04:56 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 09, 2026 2:04:56 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 09, 2026 2:04:56 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 09, 2026 2:04:56 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 09, 2026 2:04:56 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 09, 2026 2:04:56 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 09, 2026 2:04:57 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260809 02:04:57.608927 1795 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.1.192.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.1.192.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.1.192.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/krb5kdc/rangerkms_127.1.192.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.1.192.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/krb5kdc/rangerkms_127.1.192.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.1.192.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.1.192.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.1.192.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/krb5kdc/HTTP_127.1.192.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.1.192.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/krb5kdc/HTTP_127.1.192.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 09 02:04:57 dist-test-slave-ss4n krb5kdc[14725](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241097, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260809 02:04:57.707789 1795 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260809 02:04:57.718760 1795 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260809 02:04:58.720647 1795 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260809 02:04:58.720750 1795 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/ranger-kms
I20260809 02:04:58.720777 1795 mini_ranger_kms.cc:227] Using postgres at 127.1.192.212:45581
2026-08-09 02:04:58,911 [I] DB FLAVOR :POSTGRES
2026-08-09 02:04:58,911 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 02:04:58,911 [I] Checking connection
2026-08-09 02:04:58,911 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRWTcX9/build/debug/bin/postgresql.jar:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.192.212:45581/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-09 02:04:59,154 [I] connection success
2026-08-09 02:04:59,154 [I] --------- Verifying Ranger DB tables ---------
2026-08-09 02:04:59,154 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-09 02:04:59,155 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRWTcX9/build/debug/bin/postgresql.jar:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.192.212:45581/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-09 02:04:59,422 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-09 02:04:59,423 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 02:04:59,423 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-09 02:04:59,423 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRWTcX9/build/debug/bin/postgresql.jar:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.192.212:45581/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-09 02:04:59,707 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260809 02:05:00.294734 1795 mini_ranger_kms.cc:310] Created kms service
I20260809 02:05:00.505138 1795 mini_ranger_kms.cc:326] Created kudu user
I20260809 02:05:00.556672 1795 mini_ranger_kms.cc:343] Created rangerkms user
Aug 09 02:05:00 dist-test-slave-ss4n krb5kdc[14725](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241097, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.1.192.212@KRBTEST.COM
I20260809 02:05:00.998135 1795 mini_ranger_kms.cc:384] Added ranger policy
I20260809 02:05:00.998296 1795 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/ranger-kms:/tmp/dist-test-taskRWTcX9/build/debug/bin/postgresql.jar:/tmp/dist-test-taskRWTcX9/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskRWTcX9/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260809 02:05:00.998317 1795 mini_ranger_kms.cc:240] Using host: 127.1.192.212
I20260809 02:05:01.003888 1795 mini_ranger_kms.cc:276] Ranger KMS PID: 15144
I20260809 02:05:01.004000 1795 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.1.192.212:33417
Aug 09, 2026 2:05:02 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 2:05:02 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 2:05:02 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 09, 2026 2:05:02 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 09, 2026 2:05:02 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/ranger-kms/ews/webapp].
Aug 09, 2026 2:05:02 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260809 02:05:03.356843 1795 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.649s user 0.002s sys 0.175s
I20260809 02:05:03.357014 1795 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260809 02:05:03.357126 1795 mini_ranger_kms.cc:401] 127.1.192.212:33417/kms/v1/keys
Aug 09 02:05:03 dist-test-slave-ss4n krb5kdc[14725](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786241103, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.192.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 09 02:05:03 dist-test-slave-ss4n krb5kdc[14725](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241103, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.192.212@KRBTEST.COM for HTTP/127.1.192.212@KRBTEST.COM
Aug 09 02:05:04 dist-test-slave-ss4n krb5kdc[14725](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241103, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.192.212@KRBTEST.COM for HTTP/127.1.192.212@KRBTEST.COM
Aug 09 02:05:06 dist-test-slave-ss4n krb5kdc[14725](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241106, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.1.192.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.192.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.192.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab.
Entry for principal kudu/127.1.192.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab.
I20260809 02:05:06.621183 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.192.254:37883
--webserver_interface=127.1.192.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab
--principal=kudu/127.1.192.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:46275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.192.254:37883
--ranger_config_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.192.212:33417/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:05:06.740775 15197 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:05:06.741134 15197 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:05:06.741228 15197 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:05:06.745568 15197 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:05:06.745676 15197 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:05:06.745718 15197 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:05:06.745760 15197 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:05:06.745796 15197 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:05:06.752521 15197 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:46275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/master-0/wal
--ranger_kms_url=127.1.192.212:33417/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.1.192.254:37883
--ranger_config_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab
--principal=kudu/127.1.192.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.192.254:37883
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.192.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15197
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:05:06.753923 15197 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:05:06.755110 15197 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:05:06.764983 15205 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:05:06.764966 15203 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:05:06.764966 15202 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:05:06.765406 15197 server_base.cc:1034] running on GCE node
I20260809 02:05:06.766168 15197 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:05:06.767689 15197 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:05:06.768910 15197 hybrid_clock.cc:648] HybridClock initialized: now 1786241106768869 us; error 44 us; skew 500 ppm
Aug 09 02:05:06 dist-test-slave-ss4n krb5kdc[14725](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241106, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:05:06.772646 15197 init.cc:377] Logged in from keytab as kudu/127.1.192.254@KRBTEST.COM (short username kudu)
I20260809 02:05:06.773665 15197 webserver.cc:460] Webserver started at http://127.1.192.254:37955/ using document root <none> and password file <none>
I20260809 02:05:06.774230 15197 fs_manager.cc:359] Metadata directory not provided
I20260809 02:05:06.774304 15197 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:05:06.774529 15197 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 02:05:06 dist-test-slave-ss4n krb5kdc[14725](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241106, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.254@KRBTEST.COM for HTTP/127.1.192.212@KRBTEST.COM
I20260809 02:05:06.828773 15197 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/master-0/data/instance:
uuid: "c08cd059ce0e458ca8d1f1a00f024e0c"
format_stamp: "Formatted at 2026-08-09 02:05:06 on dist-test-slave-ss4n"
server_key: "84c84e9bb08b61de1719c7513474374d"
server_key_iv: "e927745b8d675f86bf5706c91341138f"
server_key_version: "kuduclusterkey@0"
I20260809 02:05:06.829425 15197 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/master-0/wal/instance:
uuid: "c08cd059ce0e458ca8d1f1a00f024e0c"
format_stamp: "Formatted at 2026-08-09 02:05:06 on dist-test-slave-ss4n"
server_key: "84c84e9bb08b61de1719c7513474374d"
server_key_iv: "e927745b8d675f86bf5706c91341138f"
server_key_version: "kuduclusterkey@0"
I20260809 02:05:06.833550 15197 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260809 02:05:06.848258 15213 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:05:06.849740 15197 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 02:05:06.849879 15197 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/master-0/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/master-0/wal
uuid: "c08cd059ce0e458ca8d1f1a00f024e0c"
format_stamp: "Formatted at 2026-08-09 02:05:06 on dist-test-slave-ss4n"
server_key: "84c84e9bb08b61de1719c7513474374d"
server_key_iv: "e927745b8d675f86bf5706c91341138f"
server_key_version: "kuduclusterkey@0"
I20260809 02:05:06.850045 15197 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:05:06.864046 15197 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:05:06.867244 15197 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:05:06.867746 15197 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:05:06.876045 15197 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.254:37883
I20260809 02:05:06.876048 15269 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.254:37883 every 8 connection(s)
I20260809 02:05:06.877103 15197 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
I20260809 02:05:06.880041 15270 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:05:06.880692 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 15197
I20260809 02:05:06.880838 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/master-0/wal/instance
I20260809 02:05:06.881161 1795 external_mini_cluster.cc:1428] Setting key aee264b19aa14bf43d33ed7b1e5e1d67
I20260809 02:05:06.887527 15270 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c08cd059ce0e458ca8d1f1a00f024e0c: Bootstrap starting.
I20260809 02:05:06.890511 15270 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c08cd059ce0e458ca8d1f1a00f024e0c: Neither blocks nor log segments found. Creating new log.
I20260809 02:05:06.891316 15270 log.cc:824] T 00000000000000000000000000000000 P c08cd059ce0e458ca8d1f1a00f024e0c: Log is configured to *not* fsync() on all Append() calls
Aug 09 02:05:06 dist-test-slave-ss4n krb5kdc[14725](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241106, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.192.254@KRBTEST.COM
I20260809 02:05:06.893811 15270 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c08cd059ce0e458ca8d1f1a00f024e0c: No bootstrap required, opened a new log
I20260809 02:05:06.897423 15270 raft_consensus.cc:348] T 00000000000000000000000000000000 P c08cd059ce0e458ca8d1f1a00f024e0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c08cd059ce0e458ca8d1f1a00f024e0c" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 37883 } }
I20260809 02:05:06.897800 15270 raft_consensus.cc:374] T 00000000000000000000000000000000 P c08cd059ce0e458ca8d1f1a00f024e0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:05:06.897881 15270 raft_consensus.cc:729] T 00000000000000000000000000000000 P c08cd059ce0e458ca8d1f1a00f024e0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c08cd059ce0e458ca8d1f1a00f024e0c, State: Initialized, Role: FOLLOWER
I20260809 02:05:06.898432 15270 consensus_queue.cc:260] T 00000000000000000000000000000000 P c08cd059ce0e458ca8d1f1a00f024e0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c08cd059ce0e458ca8d1f1a00f024e0c" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 37883 } }
I20260809 02:05:06.898623 15270 raft_consensus.cc:388] T 00000000000000000000000000000000 P c08cd059ce0e458ca8d1f1a00f024e0c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:05:06.898705 15270 raft_consensus.cc:482] T 00000000000000000000000000000000 P c08cd059ce0e458ca8d1f1a00f024e0c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:05:06.898833 15270 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c08cd059ce0e458ca8d1f1a00f024e0c [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:05:06.899096 15274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:06.882743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33304 (local address 127.1.192.254:37883)
0809 02:05:06.883233 (+ 490us) server_negotiation.cc:206] Beginning negotiation
0809 02:05:06.883254 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0809 02:05:06.883300 (+ 46us) server_negotiation.cc:407] Connection header received
0809 02:05:06.884159 (+ 859us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:05:06.884182 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:05:06.884602 (+ 420us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:05:06.884962 (+ 360us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:05:06.885886 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:06.887489 (+ 1603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:06.888652 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:06.889366 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:06.889589 (+ 223us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:05:06.892766 (+ 3177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:05:06.892773 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:05:06.892792 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:05:06.892824 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:05:06.895676 (+ 2852us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:05:06.896234 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:05:06.896240 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:05:06.896247 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:05:06.896387 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:05:06.896720 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:05:06.896723 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:05:06.896725 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:05:06.897265 (+ 540us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:05:06.897471 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0809 02:05:06.897802 (+ 331us) server_negotiation.cc:299] Negotiation successful
0809 02:05:06.898095 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":332,"thread_start_us":167,"threads_started":1}
I20260809 02:05:06.899906 15270 raft_consensus.cc:504] T 00000000000000000000000000000000 P c08cd059ce0e458ca8d1f1a00f024e0c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c08cd059ce0e458ca8d1f1a00f024e0c" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 37883 } }
I20260809 02:05:06.900293 15270 leader_election.cc:302] T 00000000000000000000000000000000 P c08cd059ce0e458ca8d1f1a00f024e0c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c08cd059ce0e458ca8d1f1a00f024e0c; no voters:
I20260809 02:05:06.900730 15270 leader_election.cc:288] T 00000000000000000000000000000000 P c08cd059ce0e458ca8d1f1a00f024e0c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:05:06.900820 15276 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c08cd059ce0e458ca8d1f1a00f024e0c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:05:06.901108 15276 raft_consensus.cc:686] T 00000000000000000000000000000000 P c08cd059ce0e458ca8d1f1a00f024e0c [term 1 LEADER]: Becoming Leader. State: Replica: c08cd059ce0e458ca8d1f1a00f024e0c, State: Running, Role: LEADER
I20260809 02:05:06.901973 15276 consensus_queue.cc:237] T 00000000000000000000000000000000 P c08cd059ce0e458ca8d1f1a00f024e0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c08cd059ce0e458ca8d1f1a00f024e0c" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 37883 } }
I20260809 02:05:06.902096 15270 sys_catalog.cc:564] T 00000000000000000000000000000000 P c08cd059ce0e458ca8d1f1a00f024e0c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:05:06.904639 15270 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260809 02:05:06.908126 15277 sys_catalog.cc:455] T 00000000000000000000000000000000 P c08cd059ce0e458ca8d1f1a00f024e0c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c08cd059ce0e458ca8d1f1a00f024e0c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c08cd059ce0e458ca8d1f1a00f024e0c" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 37883 } } }
I20260809 02:05:06.908463 15278 sys_catalog.cc:455] T 00000000000000000000000000000000 P c08cd059ce0e458ca8d1f1a00f024e0c [sys.catalog]: SysCatalogTable state changed. Reason: New leader c08cd059ce0e458ca8d1f1a00f024e0c. Latest consensus state: current_term: 1 leader_uuid: "c08cd059ce0e458ca8d1f1a00f024e0c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c08cd059ce0e458ca8d1f1a00f024e0c" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 37883 } } }
I20260809 02:05:06.908669 15277 sys_catalog.cc:458] T 00000000000000000000000000000000 P c08cd059ce0e458ca8d1f1a00f024e0c [sys.catalog]: This master's current role is: LEADER
I20260809 02:05:06.908669 15278 sys_catalog.cc:458] T 00000000000000000000000000000000 P c08cd059ce0e458ca8d1f1a00f024e0c [sys.catalog]: This master's current role is: LEADER
I20260809 02:05:06.909570 15295 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:05:06.913830 15295 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:05:06.920431 15295 catalog_manager.cc:1347] Generated new cluster ID: 1e38a0be57ac4726b5f78c9cea9dcbe3
I20260809 02:05:06.920503 15295 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:05:06.931470 15295 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:05:06.932873 15295 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:05:06.938675 15295 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c08cd059ce0e458ca8d1f1a00f024e0c: Generated new TSK 0
I20260809 02:05:06.939594 15295 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.192.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.192.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.192.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.193.keytab.
Entry for principal kudu/127.1.192.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.193.keytab.
I20260809 02:05:07.805896 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.192.212:33417/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.192.193:0
--local_ip_for_outbound_sockets=127.1.192.193
--webserver_interface=127.1.192.193
--webserver_port=0
--tserver_master_addrs=127.1.192.254:37883
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.193.keytab
--principal=kudu/127.1.192.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:46275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:05:07.971683 15318 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:05:07.972158 15318 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:05:07.972275 15318 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:05:07.978397 15318 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:05:07.978549 15318 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:05:07.978698 15318 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.192.193
I20260809 02:05:07.987534 15318 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:46275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.1.192.212:33417/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.193.keytab
--principal=kudu/127.1.192.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.192.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.192.193
--webserver_port=0
--tserver_master_addrs=127.1.192.254:37883
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15318
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.192.193
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:05:07.989392 15318 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:05:07.990861 15318 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:05:08.004235 15325 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:05:08.004360 15329 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:05:08.004571 15327 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:05:08.004626 15318 server_base.cc:1034] running on GCE node
I20260809 02:05:08.005066 15318 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:05:08.005882 15318 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:05:08.007109 15318 hybrid_clock.cc:648] HybridClock initialized: now 1786241108007034 us; error 92 us; skew 500 ppm
Aug 09 02:05:08 dist-test-slave-ss4n krb5kdc[14725](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241108, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:05:08.010121 15318 init.cc:377] Logged in from keytab as kudu/127.1.192.193@KRBTEST.COM (short username kudu)
I20260809 02:05:08.011253 15318 webserver.cc:460] Webserver started at http://127.1.192.193:33905/ using document root <none> and password file <none>
I20260809 02:05:08.011837 15318 fs_manager.cc:359] Metadata directory not provided
I20260809 02:05:08.011929 15318 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:05:08.012182 15318 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 02:05:08 dist-test-slave-ss4n krb5kdc[14725](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241108, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.193@KRBTEST.COM for HTTP/127.1.192.212@KRBTEST.COM
I20260809 02:05:08.033550 15318 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/ts-0/data/instance:
uuid: "c1654b360ae746338fafa519680cd83c"
format_stamp: "Formatted at 2026-08-09 02:05:08 on dist-test-slave-ss4n"
server_key: "82b4146aad8ed1a7e1aa0ff6f6e1036e"
server_key_iv: "325f8c8bc75363a931d80ba5f1c2b19f"
server_key_version: "kuduclusterkey@0"
I20260809 02:05:08.034301 15318 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/ts-0/wal/instance:
uuid: "c1654b360ae746338fafa519680cd83c"
format_stamp: "Formatted at 2026-08-09 02:05:08 on dist-test-slave-ss4n"
server_key: "82b4146aad8ed1a7e1aa0ff6f6e1036e"
server_key_iv: "325f8c8bc75363a931d80ba5f1c2b19f"
server_key_version: "kuduclusterkey@0"
I20260809 02:05:08.038704 15318 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 02:05:08.061851 15336 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:05:08.064777 15318 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.001s sys 0.000s
I20260809 02:05:08.064965 15318 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/ts-0/wal
uuid: "c1654b360ae746338fafa519680cd83c"
format_stamp: "Formatted at 2026-08-09 02:05:08 on dist-test-slave-ss4n"
server_key: "82b4146aad8ed1a7e1aa0ff6f6e1036e"
server_key_iv: "325f8c8bc75363a931d80ba5f1c2b19f"
server_key_version: "kuduclusterkey@0"
I20260809 02:05:08.065135 15318 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:05:08.080559 15318 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:05:08.084411 15318 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:05:08.084700 15318 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:05:08.085455 15318 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:05:08.086596 15318 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:05:08.086712 15318 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:05:08.086856 15318 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:05:08.086937 15318 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:05:08.098719 15318 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.193:36407
I20260809 02:05:08.098804 15458 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.193:36407 every 8 connection(s)
I20260809 02:05:08.099689 15318 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/ts-0/data/info.pb
I20260809 02:05:08.109439 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 15318
I20260809 02:05:08.109644 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/ts-0/wal/instance
I20260809 02:05:08.109994 1795 external_mini_cluster.cc:1428] Setting key a89e3e4087a4fb8dcb8025dcdccb2944
Aug 09 02:05:08 dist-test-slave-ss4n krb5kdc[14725](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241108, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.193@KRBTEST.COM for kudu/127.1.192.254@KRBTEST.COM
I20260809 02:05:08.119266 15461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:08.102231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.193:33715 (local address 127.1.192.254:37883)
0809 02:05:08.102504 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0809 02:05:08.102509 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:05:08.103222 (+ 713us) server_negotiation.cc:407] Connection header received
0809 02:05:08.104343 (+ 1121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:05:08.104346 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:05:08.104407 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:05:08.104504 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:05:08.105590 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:08.106153 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:08.107545 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:08.107891 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:08.107991 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:05:08.114533 (+ 6542us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:05:08.114537 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:05:08.114540 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:05:08.114569 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:05:08.116536 (+ 1967us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:05:08.117258 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:05:08.117262 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:05:08.117263 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:05:08.117357 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:05:08.117832 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:05:08.117835 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:05:08.117837 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:05:08.118095 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:05:08.118176 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 02:05:08.118932 (+ 756us) server_negotiation.cc:299] Negotiation successful
0809 02:05:08.119079 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":79,"threads_started":1}
I20260809 02:05:08.121199 15462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:08.102528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:37883 (local address 127.1.192.193:33715)
0809 02:05:08.103055 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:08.103108 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 02:05:08.104141 (+ 1033us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:05:08.104673 (+ 532us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:05:08.104680 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:05:08.104943 (+ 263us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:05:08.105416 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:05:08.105429 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:05:08.106279 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:05:08.106285 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:05:08.107396 (+ 1111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:05:08.107408 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:05:08.108211 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:05:08.108215 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:05:08.108415 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:05:08.109102 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:05:08.109124 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:05:08.114315 (+ 5191us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:05:08.116686 (+ 2371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:05:08.116693 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:05:08.116711 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:05:08.117134 (+ 423us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:05:08.117469 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:05:08.117471 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:05:08.117474 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:05:08.117713 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:05:08.118270 (+ 557us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:05:08.118276 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:05:08.118643 (+ 367us) client_negotiation.cc:769] Sending connection context
0809 02:05:08.119740 (+ 1097us) client_negotiation.cc:240] Negotiation successful
0809 02:05:08.120120 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":162,"threads_started":1}
I20260809 02:05:08.122913 15459 heartbeater.cc:347] Connected to a master server at 127.1.192.254:37883
I20260809 02:05:08.123164 15459 heartbeater.cc:464] Registering TS with master...
I20260809 02:05:08.123728 15459 heartbeater.cc:511] Master 127.1.192.254:37883 requested a full tablet report, sending...
I20260809 02:05:08.127199 15232 ts_manager.cc:194] Registered new tserver with Master: c1654b360ae746338fafa519680cd83c (127.1.192.193:36407)
I20260809 02:05:08.128765 15232 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.192.193@KRBTEST.COM'} at 127.1.192.193:33715
WARNING: no policy specified for kudu/127.1.192.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.192.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.192.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.194.keytab.
Entry for principal kudu/127.1.192.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.194.keytab.
I20260809 02:05:08.164911 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.192.212:33417/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.192.194:0
--local_ip_for_outbound_sockets=127.1.192.194
--webserver_interface=127.1.192.194
--webserver_port=0
--tserver_master_addrs=127.1.192.254:37883
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.194.keytab
--principal=kudu/127.1.192.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:46275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 02:05:08 dist-test-slave-ss4n krb5kdc[14725](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786241108, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260809 02:05:08.320092 15465 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:05:08.320556 15465 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:05:08.320685 15465 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:05:08.327018 15465 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:05:08.327189 15465 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:05:08.327332 15465 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.192.194
I20260809 02:05:08.335543 15465 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:46275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.1.192.212:33417/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.194.keytab
--principal=kudu/127.1.192.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.192.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.192.194
--webserver_port=0
--tserver_master_addrs=127.1.192.254:37883
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15465
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.192.194
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:05:08.337242 15465 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:05:08.338701 15465 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:05:08.351940 15474 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:05:08.352543 15476 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:05:08.353261 15473 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:05:08.354022 15465 server_base.cc:1034] running on GCE node
I20260809 02:05:08.354727 15465 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:05:08.355577 15465 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:05:08.356815 15465 hybrid_clock.cc:648] HybridClock initialized: now 1786241108356741 us; error 88 us; skew 500 ppm
Aug 09 02:05:08 dist-test-slave-ss4n krb5kdc[14725](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241108, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:05:08.360611 15465 init.cc:377] Logged in from keytab as kudu/127.1.192.194@KRBTEST.COM (short username kudu)
I20260809 02:05:08.361837 15465 webserver.cc:460] Webserver started at http://127.1.192.194:36605/ using document root <none> and password file <none>
I20260809 02:05:08.362610 15465 fs_manager.cc:359] Metadata directory not provided
I20260809 02:05:08.362758 15465 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:05:08.363051 15465 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 02:05:08 dist-test-slave-ss4n krb5kdc[14725](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241108, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.194@KRBTEST.COM for HTTP/127.1.192.212@KRBTEST.COM
I20260809 02:05:08.379958 15465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/ts-1/data/instance:
uuid: "3ff9b9358080455d855a18628c1952fc"
format_stamp: "Formatted at 2026-08-09 02:05:08 on dist-test-slave-ss4n"
server_key: "b62d9300baacee52fa43db139f349a00"
server_key_iv: "950977d467f86c4b50201385e6894ba1"
server_key_version: "kuduclusterkey@0"
I20260809 02:05:08.382644 15465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/ts-1/wal/instance:
uuid: "3ff9b9358080455d855a18628c1952fc"
format_stamp: "Formatted at 2026-08-09 02:05:08 on dist-test-slave-ss4n"
server_key: "b62d9300baacee52fa43db139f349a00"
server_key_iv: "950977d467f86c4b50201385e6894ba1"
server_key_version: "kuduclusterkey@0"
I20260809 02:05:08.390470 15465 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.004s sys 0.000s
I20260809 02:05:08.422462 15484 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:05:08.424263 15465 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 02:05:08.424405 15465 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/ts-1/wal
uuid: "3ff9b9358080455d855a18628c1952fc"
format_stamp: "Formatted at 2026-08-09 02:05:08 on dist-test-slave-ss4n"
server_key: "b62d9300baacee52fa43db139f349a00"
server_key_iv: "950977d467f86c4b50201385e6894ba1"
server_key_version: "kuduclusterkey@0"
I20260809 02:05:08.424551 15465 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:05:08.460013 15465 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:05:08.464162 15465 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:05:08.464433 15465 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:05:08.465291 15465 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:05:08.466418 15465 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:05:08.466543 15465 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:05:08.466652 15465 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:05:08.466718 15465 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:05:08.478128 15465 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.194:36917
I20260809 02:05:08.479529 15465 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/ts-1/data/info.pb
I20260809 02:05:08.488917 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 15465
I20260809 02:05:08.489045 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/ts-1/wal/instance
I20260809 02:05:08.489368 1795 external_mini_cluster.cc:1428] Setting key 9c07b92a9086c478d069f139b51eb02a
I20260809 02:05:08.512642 15605 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.194:36917 every 8 connection(s)
WARNING: no policy specified for kudu/127.1.192.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.192.195@KRBTEST.COM" created.
Aug 09 02:05:08 dist-test-slave-ss4n krb5kdc[14725](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241108, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.194@KRBTEST.COM for kudu/127.1.192.254@KRBTEST.COM
I20260809 02:05:08.534360 15461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:08.512427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.194:33069 (local address 127.1.192.254:37883)
0809 02:05:08.513417 (+ 990us) server_negotiation.cc:206] Beginning negotiation
0809 02:05:08.513422 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:05:08.520013 (+ 6591us) server_negotiation.cc:407] Connection header received
0809 02:05:08.521409 (+ 1396us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:05:08.521413 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:05:08.521511 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:05:08.521570 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:05:08.523422 (+ 1852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:08.524247 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:08.526060 (+ 1813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:08.526399 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:08.526490 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:05:08.529781 (+ 3291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:05:08.529785 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:05:08.529788 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:05:08.529822 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:05:08.531663 (+ 1841us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:05:08.532341 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:05:08.532346 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:05:08.532347 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:05:08.532445 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:05:08.532891 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:05:08.532895 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:05:08.532897 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:05:08.533134 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:05:08.533223 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 02:05:08.534039 (+ 816us) server_negotiation.cc:299] Negotiation successful
0809 02:05:08.534183 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":884}
I20260809 02:05:08.536034 15609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:08.512224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:37883 (local address 127.1.192.194:33069)
0809 02:05:08.519769 (+ 7545us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:08.519864 (+ 95us) client_negotiation.cc:174] Beginning negotiation
0809 02:05:08.521117 (+ 1253us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:05:08.522038 (+ 921us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:05:08.522049 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:05:08.522483 (+ 434us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:05:08.523234 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:05:08.523255 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:05:08.524421 (+ 1166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:05:08.524424 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:05:08.525893 (+ 1469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:05:08.525904 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:05:08.526716 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:05:08.526720 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:05:08.526925 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:05:08.527689 (+ 764us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:05:08.527711 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:05:08.529559 (+ 1848us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:05:08.531821 (+ 2262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:05:08.531828 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:05:08.531846 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:05:08.532212 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:05:08.532578 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:05:08.532581 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:05:08.532583 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:05:08.532775 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:05:08.533295 (+ 520us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:05:08.533301 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:05:08.533740 (+ 439us) client_negotiation.cc:769] Sending connection context
0809 02:05:08.534562 (+ 822us) client_negotiation.cc:240] Negotiation successful
0809 02:05:08.534939 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7226,"thread_start_us":254,"threads_started":1}
I20260809 02:05:08.537578 15606 heartbeater.cc:347] Connected to a master server at 127.1.192.254:37883
I20260809 02:05:08.537789 15606 heartbeater.cc:464] Registering TS with master...
I20260809 02:05:08.538419 15606 heartbeater.cc:511] Master 127.1.192.254:37883 requested a full tablet report, sending...
I20260809 02:05:08.539870 15232 ts_manager.cc:194] Registered new tserver with Master: 3ff9b9358080455d855a18628c1952fc (127.1.192.194:36917)
I20260809 02:05:08.540725 15232 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.192.194@KRBTEST.COM'} at 127.1.192.194:33069
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.192.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.195.keytab.
Entry for principal kudu/127.1.192.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.195.keytab.
I20260809 02:05:08.561069 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.192.212:33417/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.192.195:0
--local_ip_for_outbound_sockets=127.1.192.195
--webserver_interface=127.1.192.195
--webserver_port=0
--tserver_master_addrs=127.1.192.254:37883
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.195.keytab
--principal=kudu/127.1.192.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:46275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:05:08.728335 15611 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:05:08.729249 15611 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:05:08.729418 15611 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:05:08.737995 15611 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:05:08.738188 15611 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:05:08.738379 15611 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.192.195
I20260809 02:05:08.747687 15611 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:46275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.1.192.212:33417/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.195.keytab
--principal=kudu/127.1.192.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.192.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.192.195
--webserver_port=0
--tserver_master_addrs=127.1.192.254:37883
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15611
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.192.195
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:05:08.749640 15611 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:05:08.751274 15611 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:05:08.767254 15618 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:05:08.767704 15617 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:05:08.767256 15620 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:05:08.770255 15611 server_base.cc:1034] running on GCE node
I20260809 02:05:08.770920 15611 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:05:08.771766 15611 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:05:08.772945 15611 hybrid_clock.cc:648] HybridClock initialized: now 1786241108772865 us; error 102 us; skew 500 ppm
Aug 09 02:05:08 dist-test-slave-ss4n krb5kdc[14725](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241108, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:05:08.777230 15611 init.cc:377] Logged in from keytab as kudu/127.1.192.195@KRBTEST.COM (short username kudu)
I20260809 02:05:08.778626 15611 webserver.cc:460] Webserver started at http://127.1.192.195:42975/ using document root <none> and password file <none>
I20260809 02:05:08.779506 15611 fs_manager.cc:359] Metadata directory not provided
I20260809 02:05:08.779678 15611 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:05:08.780010 15611 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 02:05:08 dist-test-slave-ss4n krb5kdc[14725](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241108, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.195@KRBTEST.COM for HTTP/127.1.192.212@KRBTEST.COM
I20260809 02:05:08.799463 15611 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/ts-2/data/instance:
uuid: "df9f9d6c29894056bc556be4ff9992ba"
format_stamp: "Formatted at 2026-08-09 02:05:08 on dist-test-slave-ss4n"
server_key: "104574bb35ef9c4e44af89139ecf6b8e"
server_key_iv: "fac88f366fba47ee418c63f975bfa994"
server_key_version: "kuduclusterkey@0"
I20260809 02:05:08.801180 15611 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/ts-2/wal/instance:
uuid: "df9f9d6c29894056bc556be4ff9992ba"
format_stamp: "Formatted at 2026-08-09 02:05:08 on dist-test-slave-ss4n"
server_key: "104574bb35ef9c4e44af89139ecf6b8e"
server_key_iv: "fac88f366fba47ee418c63f975bfa994"
server_key_version: "kuduclusterkey@0"
I20260809 02:05:08.807538 15611 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.003s
I20260809 02:05:08.827721 15627 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:05:08.829891 15611 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.003s sys 0.000s
I20260809 02:05:08.830068 15611 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/ts-2/wal
uuid: "df9f9d6c29894056bc556be4ff9992ba"
format_stamp: "Formatted at 2026-08-09 02:05:08 on dist-test-slave-ss4n"
server_key: "104574bb35ef9c4e44af89139ecf6b8e"
server_key_iv: "fac88f366fba47ee418c63f975bfa994"
server_key_version: "kuduclusterkey@0"
I20260809 02:05:08.830247 15611 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:05:08.887634 15611 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:05:08.892057 15611 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:05:08.892393 15611 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:05:08.893317 15611 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:05:08.894635 15611 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:05:08.894759 15611 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:05:08.894881 15611 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:05:08.894958 15611 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:05:08.911855 15611 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.195:34419
I20260809 02:05:08.913275 15611 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/ts-2/data/info.pb
I20260809 02:05:08.914574 15747 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.195:34419 every 8 connection(s)
Aug 09 02:05:08 dist-test-slave-ss4n krb5kdc[14725](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241108, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.254@KRBTEST.COM for HTTP/127.1.192.212@KRBTEST.COM
I20260809 02:05:08.921095 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 15611
I20260809 02:05:08.921243 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/ts-2/wal/instance
I20260809 02:05:08.921648 1795 external_mini_cluster.cc:1428] Setting key 3a6f5e911fc5b6646e85a339b4e541a4
Aug 09 02:05:08 dist-test-slave-ss4n krb5kdc[14725](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241108, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.195@KRBTEST.COM for kudu/127.1.192.254@KRBTEST.COM
I20260809 02:05:08.936267 15461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:08.919186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.195:35993 (local address 127.1.192.254:37883)
0809 02:05:08.919335 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0809 02:05:08.919340 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:05:08.920219 (+ 879us) server_negotiation.cc:407] Connection header received
0809 02:05:08.921907 (+ 1688us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:05:08.921911 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:05:08.921976 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:05:08.922048 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:05:08.923939 (+ 1891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:08.924852 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:08.926816 (+ 1964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:08.927184 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:08.927276 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:05:08.931012 (+ 3736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:05:08.931016 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:05:08.931020 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:05:08.931052 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:05:08.933126 (+ 2074us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:05:08.933930 (+ 804us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:05:08.933934 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:05:08.933936 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:05:08.934037 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:05:08.934654 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:05:08.934659 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:05:08.934661 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:05:08.935040 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:05:08.935136 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0809 02:05:08.935931 (+ 795us) server_negotiation.cc:299] Negotiation successful
0809 02:05:08.936074 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 02:05:08.938431 15750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:08.917887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:37883 (local address 127.1.192.195:35993)
0809 02:05:08.920026 (+ 2139us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:08.920096 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0809 02:05:08.921384 (+ 1288us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:05:08.922549 (+ 1165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:05:08.922559 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:05:08.922964 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:05:08.923753 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:05:08.923778 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:05:08.925040 (+ 1262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:05:08.925044 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:05:08.926651 (+ 1607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:05:08.926666 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:05:08.927538 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:05:08.927543 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:05:08.927779 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:05:08.928653 (+ 874us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:05:08.928678 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:05:08.930773 (+ 2095us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:05:08.933304 (+ 2531us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:05:08.933312 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:05:08.933332 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:05:08.933793 (+ 461us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:05:08.934162 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:05:08.934165 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:05:08.934168 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:05:08.934505 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:05:08.935215 (+ 710us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:05:08.935224 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:05:08.935613 (+ 389us) client_negotiation.cc:769] Sending connection context
0809 02:05:08.936811 (+ 1198us) client_negotiation.cc:240] Negotiation successful
0809 02:05:08.937211 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1889,"thread_start_us":193,"threads_started":1}
I20260809 02:05:08.940132 15748 heartbeater.cc:347] Connected to a master server at 127.1.192.254:37883
I20260809 02:05:08.940402 15748 heartbeater.cc:464] Registering TS with master...
I20260809 02:05:08.941071 15748 heartbeater.cc:511] Master 127.1.192.254:37883 requested a full tablet report, sending...
I20260809 02:05:08.942458 15232 ts_manager.cc:194] Registered new tserver with Master: df9f9d6c29894056bc556be4ff9992ba (127.1.192.195:34419)
I20260809 02:05:08.943322 15232 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.192.195@KRBTEST.COM'} at 127.1.192.195:35993
I20260809 02:05:08.954341 1795 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 02:05:08.955101 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 15318
I20260809 02:05:08.968662 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 15465
I20260809 02:05:08.985085 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 15611
I20260809 02:05:08.996326 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 15197
I20260809 02:05:09.006955 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.192.254:37883
--webserver_interface=127.1.192.254
--webserver_port=37955
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab
--principal=kudu/127.1.192.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:46275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.192.254:37883
--ranger_config_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.192.212:33417/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.1.192.212:33417/kms with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:05:09.133997 15751 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:05:09.134325 15751 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:05:09.134402 15751 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:05:09.138474 15751 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:05:09.138542 15751 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:05:09.138562 15751 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:05:09.138578 15751 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:05:09.138602 15751 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:05:09.144114 15751 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:46275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.1.192.212:33417/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.1.192.254:37883
--ranger_config_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab
--principal=kudu/127.1.192.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.192.254:37883
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.192.254
--webserver_port=37955
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15751
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:05:09.145191 15751 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:05:09.146193 15751 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:05:09.155020 15757 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:05:09.154985 15758 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:05:09.155164 15751 server_base.cc:1034] running on GCE node
W20260809 02:05:09.154985 15760 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:05:09.156036 15751 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:05:09.157191 15751 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:05:09.158371 15751 hybrid_clock.cc:648] HybridClock initialized: now 1786241109158296 us; error 90 us; skew 500 ppm
Aug 09 02:05:09 dist-test-slave-ss4n krb5kdc[14725](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241109, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:05:09.161212 15751 init.cc:377] Logged in from keytab as kudu/127.1.192.254@KRBTEST.COM (short username kudu)
I20260809 02:05:09.162346 15751 webserver.cc:460] Webserver started at http://127.1.192.254:37955/ using document root <none> and password file <none>
I20260809 02:05:09.163049 15751 fs_manager.cc:359] Metadata directory not provided
I20260809 02:05:09.163148 15751 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 02:05:09 dist-test-slave-ss4n krb5kdc[14725](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241109, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.254@KRBTEST.COM for HTTP/127.1.192.212@KRBTEST.COM
I20260809 02:05:09.215165 15751 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260809 02:05:09.217764 15770 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:05:09.219238 15751 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 02:05:09.219396 15751 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/master-0/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/master-0/wal
uuid: "c08cd059ce0e458ca8d1f1a00f024e0c"
format_stamp: "Formatted at 2026-08-09 02:05:06 on dist-test-slave-ss4n"
server_key: "84c84e9bb08b61de1719c7513474374d"
server_key_iv: "e927745b8d675f86bf5706c91341138f"
server_key_version: "kuduclusterkey@0"
I20260809 02:05:09.219991 15751 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:05:09.258901 15751 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:05:09.261845 15751 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:05:09.262066 15751 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:05:09.269847 15751 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.254:37883
I20260809 02:05:09.269912 15825 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.254:37883 every 8 connection(s)
I20260809 02:05:09.271039 15751 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
I20260809 02:05:09.275393 15826 sys_catalog.cc:263] Verifying existing consensus state
I20260809 02:05:09.276916 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 15751
I20260809 02:05:09.277163 15826 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c08cd059ce0e458ca8d1f1a00f024e0c: Bootstrap starting.
I20260809 02:05:09.277449 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.192.212:33417/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.192.193:36407
--local_ip_for_outbound_sockets=127.1.192.193
--tserver_master_addrs=127.1.192.254:37883
--webserver_port=33905
--webserver_interface=127.1.192.193
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.193.keytab
--principal=kudu/127.1.192.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:46275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.1.192.212:33417/kms with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 02:05:09.294284 15826 log.cc:824] T 00000000000000000000000000000000 P c08cd059ce0e458ca8d1f1a00f024e0c: Log is configured to *not* fsync() on all Append() calls
I20260809 02:05:09.300907 15826 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c08cd059ce0e458ca8d1f1a00f024e0c: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 02:05:09.301393 15826 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c08cd059ce0e458ca8d1f1a00f024e0c: Bootstrap complete.
I20260809 02:05:09.305758 15826 raft_consensus.cc:348] T 00000000000000000000000000000000 P c08cd059ce0e458ca8d1f1a00f024e0c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c08cd059ce0e458ca8d1f1a00f024e0c" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 37883 } }
I20260809 02:05:09.306180 15826 raft_consensus.cc:729] T 00000000000000000000000000000000 P c08cd059ce0e458ca8d1f1a00f024e0c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c08cd059ce0e458ca8d1f1a00f024e0c, State: Initialized, Role: FOLLOWER
I20260809 02:05:09.306651 15826 consensus_queue.cc:260] T 00000000000000000000000000000000 P c08cd059ce0e458ca8d1f1a00f024e0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c08cd059ce0e458ca8d1f1a00f024e0c" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 37883 } }
I20260809 02:05:09.306799 15826 raft_consensus.cc:388] T 00000000000000000000000000000000 P c08cd059ce0e458ca8d1f1a00f024e0c [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:05:09.306876 15826 raft_consensus.cc:482] T 00000000000000000000000000000000 P c08cd059ce0e458ca8d1f1a00f024e0c [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:05:09.306991 15826 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c08cd059ce0e458ca8d1f1a00f024e0c [term 1 FOLLOWER]: Advancing to term 2
I20260809 02:05:09.308908 15826 raft_consensus.cc:504] T 00000000000000000000000000000000 P c08cd059ce0e458ca8d1f1a00f024e0c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c08cd059ce0e458ca8d1f1a00f024e0c" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 37883 } }
I20260809 02:05:09.309242 15826 leader_election.cc:302] T 00000000000000000000000000000000 P c08cd059ce0e458ca8d1f1a00f024e0c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c08cd059ce0e458ca8d1f1a00f024e0c; no voters:
I20260809 02:05:09.309607 15826 leader_election.cc:288] T 00000000000000000000000000000000 P c08cd059ce0e458ca8d1f1a00f024e0c [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 02:05:09.309762 15830 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c08cd059ce0e458ca8d1f1a00f024e0c [term 2 FOLLOWER]: Leader election won for term 2
I20260809 02:05:09.310017 15830 raft_consensus.cc:686] T 00000000000000000000000000000000 P c08cd059ce0e458ca8d1f1a00f024e0c [term 2 LEADER]: Becoming Leader. State: Replica: c08cd059ce0e458ca8d1f1a00f024e0c, State: Running, Role: LEADER
I20260809 02:05:09.310396 15830 consensus_queue.cc:237] T 00000000000000000000000000000000 P c08cd059ce0e458ca8d1f1a00f024e0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c08cd059ce0e458ca8d1f1a00f024e0c" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 37883 } }
I20260809 02:05:09.310845 15826 sys_catalog.cc:564] T 00000000000000000000000000000000 P c08cd059ce0e458ca8d1f1a00f024e0c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:05:09.311967 15832 sys_catalog.cc:455] T 00000000000000000000000000000000 P c08cd059ce0e458ca8d1f1a00f024e0c [sys.catalog]: SysCatalogTable state changed. Reason: New leader c08cd059ce0e458ca8d1f1a00f024e0c. Latest consensus state: current_term: 2 leader_uuid: "c08cd059ce0e458ca8d1f1a00f024e0c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c08cd059ce0e458ca8d1f1a00f024e0c" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 37883 } } }
I20260809 02:05:09.312007 15831 sys_catalog.cc:455] T 00000000000000000000000000000000 P c08cd059ce0e458ca8d1f1a00f024e0c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "c08cd059ce0e458ca8d1f1a00f024e0c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c08cd059ce0e458ca8d1f1a00f024e0c" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 37883 } } }
I20260809 02:05:09.312090 15832 sys_catalog.cc:458] T 00000000000000000000000000000000 P c08cd059ce0e458ca8d1f1a00f024e0c [sys.catalog]: This master's current role is: LEADER
I20260809 02:05:09.312130 15831 sys_catalog.cc:458] T 00000000000000000000000000000000 P c08cd059ce0e458ca8d1f1a00f024e0c [sys.catalog]: This master's current role is: LEADER
I20260809 02:05:09.313298 15826 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260809 02:05:09.313678 15840 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:05:09.320035 15840 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:05:09.321071 15840 catalog_manager.cc:1259] Loaded cluster ID: 1e38a0be57ac4726b5f78c9cea9dcbe3
I20260809 02:05:09.321219 15840 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:05:09.324334 15840 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:05:09.325217 15840 catalog_manager.cc:5879] T 00000000000000000000000000000000 P c08cd059ce0e458ca8d1f1a00f024e0c: Loaded TSK: 0
I20260809 02:05:09.326731 15840 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 02:05:09.424250 15828 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:05:09.424722 15828 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:05:09.424855 15828 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:05:09.431022 15828 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:05:09.431164 15828 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:05:09.431288 15828 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.192.193
I20260809 02:05:09.439169 15828 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:46275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.1.192.212:33417/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.193.keytab
--principal=kudu/127.1.192.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.192.193:36407
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.192.193
--webserver_port=33905
--tserver_master_addrs=127.1.192.254:37883
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15828
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.192.193
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:05:09.440534 15828 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:05:09.441900 15828 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:05:09.454548 15867 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:05:09.455096 15868 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:05:09.455331 15870 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:05:09.456831 15828 server_base.cc:1034] running on GCE node
I20260809 02:05:09.457443 15828 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:05:09.458266 15828 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:05:09.459465 15828 hybrid_clock.cc:648] HybridClock initialized: now 1786241109459436 us; error 44 us; skew 500 ppm
Aug 09 02:05:09 dist-test-slave-ss4n krb5kdc[14725](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241109, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:05:09.463210 15828 init.cc:377] Logged in from keytab as kudu/127.1.192.193@KRBTEST.COM (short username kudu)
I20260809 02:05:09.464380 15828 webserver.cc:460] Webserver started at http://127.1.192.193:33905/ using document root <none> and password file <none>
I20260809 02:05:09.465083 15828 fs_manager.cc:359] Metadata directory not provided
I20260809 02:05:09.465188 15828 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 02:05:09 dist-test-slave-ss4n krb5kdc[14725](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241109, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.193@KRBTEST.COM for HTTP/127.1.192.212@KRBTEST.COM
I20260809 02:05:09.495685 15828 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260809 02:05:09.498986 15878 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:05:09.500677 15828 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260809 02:05:09.500828 15828 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/ts-0/wal
uuid: "c1654b360ae746338fafa519680cd83c"
format_stamp: "Formatted at 2026-08-09 02:05:08 on dist-test-slave-ss4n"
server_key: "82b4146aad8ed1a7e1aa0ff6f6e1036e"
server_key_iv: "325f8c8bc75363a931d80ba5f1c2b19f"
server_key_version: "kuduclusterkey@0"
I20260809 02:05:09.501572 15828 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:05:09.521281 15828 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:05:09.525174 15828 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:05:09.525547 15828 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:05:09.526357 15828 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:05:09.527464 15828 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:05:09.527570 15828 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:05:09.527706 15828 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:05:09.527782 15828 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:05:09.539465 15828 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.193:36407
I20260809 02:05:09.541015 15828 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/ts-0/data/info.pb
I20260809 02:05:09.541474 16002 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.193:36407 every 8 connection(s)
I20260809 02:05:09.547952 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 15828
I20260809 02:05:09.548491 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.192.212:33417/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.192.194:36917
--local_ip_for_outbound_sockets=127.1.192.194
--tserver_master_addrs=127.1.192.254:37883
--webserver_port=36605
--webserver_interface=127.1.192.194
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.194.keytab
--principal=kudu/127.1.192.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:46275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.1.192.212:33417/kms with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 02:05:09 dist-test-slave-ss4n krb5kdc[14725](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241109, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.193@KRBTEST.COM for kudu/127.1.192.254@KRBTEST.COM
I20260809 02:05:09.576936 16005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:09.544402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.193:44947 (local address 127.1.192.254:37883)
0809 02:05:09.544991 (+ 589us) server_negotiation.cc:206] Beginning negotiation
0809 02:05:09.545012 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0809 02:05:09.546836 (+ 1824us) server_negotiation.cc:407] Connection header received
0809 02:05:09.547176 (+ 340us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:05:09.547191 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:05:09.547653 (+ 462us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:05:09.548067 (+ 414us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:05:09.549913 (+ 1846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:09.551776 (+ 1863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:09.553631 (+ 1855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:09.554398 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:09.554583 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:05:09.566597 (+ 12014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:05:09.566606 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:05:09.566627 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:05:09.566672 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:05:09.569110 (+ 2438us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:05:09.572904 (+ 3794us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:05:09.572911 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:05:09.572919 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:05:09.573107 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:05:09.573967 (+ 860us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:05:09.573971 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:05:09.573973 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:05:09.574559 (+ 586us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:05:09.574704 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0809 02:05:09.575455 (+ 751us) server_negotiation.cc:299] Negotiation successful
0809 02:05:09.575806 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":429,"thread_start_us":115,"threads_started":1}
I20260809 02:05:09.577812 16006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:09.544495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:37883 (local address 127.1.192.193:44947)
0809 02:05:09.545312 (+ 817us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:09.545374 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0809 02:05:09.546488 (+ 1114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:05:09.548588 (+ 2100us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:05:09.548597 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:05:09.548972 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:05:09.549679 (+ 707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:05:09.549702 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:05:09.551944 (+ 2242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:05:09.551947 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:05:09.553457 (+ 1510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:05:09.553470 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:05:09.554843 (+ 1373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:05:09.554847 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:05:09.555061 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:05:09.561569 (+ 6508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:05:09.561598 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:05:09.566239 (+ 4641us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:05:09.569295 (+ 3056us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:05:09.569302 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:05:09.569319 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:05:09.569744 (+ 425us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:05:09.573565 (+ 3821us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:05:09.573571 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:05:09.573575 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:05:09.573830 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:05:09.574780 (+ 950us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:05:09.574787 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:05:09.575096 (+ 309us) client_negotiation.cc:769] Sending connection context
0809 02:05:09.576491 (+ 1395us) client_negotiation.cc:240] Negotiation successful
0809 02:05:09.576857 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":590,"thread_start_us":158,"threads_started":1}
I20260809 02:05:09.580777 16003 heartbeater.cc:347] Connected to a master server at 127.1.192.254:37883
I20260809 02:05:09.581019 16003 heartbeater.cc:464] Registering TS with master...
I20260809 02:05:09.581598 16003 heartbeater.cc:511] Master 127.1.192.254:37883 requested a full tablet report, sending...
W20260809 02:05:09.583891 16003 heartbeater.cc:644] Failed to heartbeat to 127.1.192.254:37883 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 02:05:09.584009 16003 heartbeater.cc:464] Registering TS with master...
I20260809 02:05:09.584177 16003 heartbeater.cc:511] Master 127.1.192.254:37883 requested a full tablet report, sending...
I20260809 02:05:09.585125 16003 heartbeater.cc:464] Registering TS with master...
I20260809 02:05:09.585294 16003 heartbeater.cc:511] Master 127.1.192.254:37883 requested a full tablet report, sending...
W20260809 02:05:09.586189 16003 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 02:05:09.705739 16007 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:05:09.706059 16007 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:05:09.706137 16007 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:05:09.711912 16007 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:05:09.712009 16007 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:05:09.712102 16007 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.192.194
I20260809 02:05:09.720450 16007 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:46275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.1.192.212:33417/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.194.keytab
--principal=kudu/127.1.192.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.192.194:36917
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.192.194
--webserver_port=36605
--tserver_master_addrs=127.1.192.254:37883
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16007
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.192.194
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:05:09.722065 16007 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:05:09.723201 16007 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:05:09.736227 16012 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:05:09.736400 16013 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:05:09.736343 16015 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:05:09.737818 16007 server_base.cc:1034] running on GCE node
I20260809 02:05:09.738364 16007 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:05:09.739233 16007 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:05:09.740433 16007 hybrid_clock.cc:648] HybridClock initialized: now 1786241109740400 us; error 46 us; skew 500 ppm
Aug 09 02:05:09 dist-test-slave-ss4n krb5kdc[14725](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241109, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:05:09.744247 16007 init.cc:377] Logged in from keytab as kudu/127.1.192.194@KRBTEST.COM (short username kudu)
I20260809 02:05:09.745560 16007 webserver.cc:460] Webserver started at http://127.1.192.194:36605/ using document root <none> and password file <none>
I20260809 02:05:09.746366 16007 fs_manager.cc:359] Metadata directory not provided
I20260809 02:05:09.746534 16007 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 02:05:09 dist-test-slave-ss4n krb5kdc[14725](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241109, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.194@KRBTEST.COM for HTTP/127.1.192.212@KRBTEST.COM
I20260809 02:05:09.772658 16007 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.003s sys 0.000s
I20260809 02:05:09.776052 16024 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:05:09.777732 16007 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260809 02:05:09.777858 16007 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/ts-1/wal
uuid: "3ff9b9358080455d855a18628c1952fc"
format_stamp: "Formatted at 2026-08-09 02:05:08 on dist-test-slave-ss4n"
server_key: "b62d9300baacee52fa43db139f349a00"
server_key_iv: "950977d467f86c4b50201385e6894ba1"
server_key_version: "kuduclusterkey@0"
I20260809 02:05:09.778627 16007 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:05:09.803712 16007 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:05:09.807782 16007 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:05:09.808089 16007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:05:09.808919 16007 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:05:09.810096 16007 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:05:09.810215 16007 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:05:09.810316 16007 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:05:09.810384 16007 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:05:09.822482 16007 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.194:36917
I20260809 02:05:09.823706 16007 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/ts-1/data/info.pb
I20260809 02:05:09.823985 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 16007
I20260809 02:05:09.822566 16146 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.194:36917 every 8 connection(s)
I20260809 02:05:09.824513 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.192.212:33417/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.192.195:34419
--local_ip_for_outbound_sockets=127.1.192.195
--tserver_master_addrs=127.1.192.254:37883
--webserver_port=42975
--webserver_interface=127.1.192.195
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.195.keytab
--principal=kudu/127.1.192.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:46275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.1.192.212:33417/kms with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 02:05:09 dist-test-slave-ss4n krb5kdc[14725](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241109, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.194@KRBTEST.COM for kudu/127.1.192.254@KRBTEST.COM
I20260809 02:05:09.861791 16005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:09.826155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.194:60485 (local address 127.1.192.254:37883)
0809 02:05:09.827875 (+ 1720us) server_negotiation.cc:206] Beginning negotiation
0809 02:05:09.827882 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 02:05:09.832420 (+ 4538us) server_negotiation.cc:407] Connection header received
0809 02:05:09.833769 (+ 1349us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:05:09.833772 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:05:09.833841 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:05:09.833928 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:05:09.846536 (+ 12608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:09.847710 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:09.850476 (+ 2766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:09.850857 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:09.850940 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:05:09.854435 (+ 3495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:05:09.854451 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:05:09.854456 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:05:09.854493 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:05:09.858911 (+ 4418us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:05:09.859674 (+ 763us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:05:09.859678 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:05:09.859680 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:05:09.859775 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:05:09.860273 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:05:09.860276 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:05:09.860278 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:05:09.860531 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:05:09.860617 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 02:05:09.861314 (+ 697us) server_negotiation.cc:299] Negotiation successful
0809 02:05:09.861535 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1590}
I20260809 02:05:09.864463 16150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:09.826707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:37883 (local address 127.1.192.194:60485)
0809 02:05:09.829947 (+ 3240us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:09.830024 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0809 02:05:09.833479 (+ 3455us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:05:09.834337 (+ 858us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:05:09.834351 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:05:09.844551 (+ 10200us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:05:09.845382 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:05:09.845408 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:05:09.848611 (+ 3203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:05:09.848615 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:05:09.850293 (+ 1678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:05:09.850308 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:05:09.851175 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:05:09.851179 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:05:09.851397 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:05:09.852147 (+ 750us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:05:09.852175 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:05:09.854175 (+ 2000us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:05:09.859090 (+ 4915us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:05:09.859097 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:05:09.859117 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:05:09.859539 (+ 422us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:05:09.859896 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:05:09.859898 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:05:09.859900 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:05:09.860090 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:05:09.860687 (+ 597us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:05:09.860693 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:05:09.860996 (+ 303us) client_negotiation.cc:769] Sending connection context
0809 02:05:09.862484 (+ 1488us) client_negotiation.cc:240] Negotiation successful
0809 02:05:09.862839 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2939,"thread_start_us":170,"threads_started":1}
I20260809 02:05:09.866112 16147 heartbeater.cc:347] Connected to a master server at 127.1.192.254:37883
I20260809 02:05:09.866367 16147 heartbeater.cc:464] Registering TS with master...
I20260809 02:05:09.866966 16147 heartbeater.cc:511] Master 127.1.192.254:37883 requested a full tablet report, sending...
W20260809 02:05:09.868753 16147 heartbeater.cc:644] Failed to heartbeat to 127.1.192.254:37883 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 02:05:09.868876 16147 heartbeater.cc:464] Registering TS with master...
I20260809 02:05:09.869078 16147 heartbeater.cc:511] Master 127.1.192.254:37883 requested a full tablet report, sending...
I20260809 02:05:09.870007 16147 heartbeater.cc:464] Registering TS with master...
I20260809 02:05:09.870175 16147 heartbeater.cc:511] Master 127.1.192.254:37883 requested a full tablet report, sending...
W20260809 02:05:09.870929 16147 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 02:05:09.987744 16149 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:05:09.988174 16149 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:05:09.988319 16149 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:05:09.994803 16149 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:05:09.994905 16149 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:05:09.995003 16149 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.192.195
I20260809 02:05:10.002642 16149 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:46275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.1.192.212:33417/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.195.keytab
--principal=kudu/127.1.192.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.192.195:34419
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.192.195
--webserver_port=42975
--tserver_master_addrs=127.1.192.254:37883
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16149
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.192.195
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:05:10.004268 16149 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:05:10.005692 16149 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:05:10.016753 16155 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:05:10.016777 16158 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:05:10.017083 16156 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:05:10.017266 16149 server_base.cc:1034] running on GCE node
I20260809 02:05:10.017758 16149 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:05:10.018639 16149 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:05:10.019833 16149 hybrid_clock.cc:648] HybridClock initialized: now 1786241110019794 us; error 52 us; skew 500 ppm
Aug 09 02:05:10 dist-test-slave-ss4n krb5kdc[14725](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241110, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:05:10.027401 16149 init.cc:377] Logged in from keytab as kudu/127.1.192.195@KRBTEST.COM (short username kudu)
I20260809 02:05:10.028589 16149 webserver.cc:460] Webserver started at http://127.1.192.195:42975/ using document root <none> and password file <none>
I20260809 02:05:10.029238 16149 fs_manager.cc:359] Metadata directory not provided
I20260809 02:05:10.029353 16149 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 02:05:10 dist-test-slave-ss4n krb5kdc[14725](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241110, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.195@KRBTEST.COM for HTTP/127.1.192.212@KRBTEST.COM
I20260809 02:05:10.050318 16149 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.000s
I20260809 02:05:10.055958 16166 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:05:10.057425 16149 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.003s sys 0.001s
I20260809 02:05:10.057628 16149 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/ts-2/wal
uuid: "df9f9d6c29894056bc556be4ff9992ba"
format_stamp: "Formatted at 2026-08-09 02:05:08 on dist-test-slave-ss4n"
server_key: "104574bb35ef9c4e44af89139ecf6b8e"
server_key_iv: "fac88f366fba47ee418c63f975bfa994"
server_key_version: "kuduclusterkey@0"
I20260809 02:05:10.058244 16149 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:05:10.087431 16149 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:05:10.091428 16149 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:05:10.091670 16149 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:05:10.092372 16149 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:05:10.093447 16149 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:05:10.093565 16149 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:05:10.093644 16149 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:05:10.093674 16149 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:05:10.105809 16149 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.195:34419
I20260809 02:05:10.105844 16292 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.195:34419 every 8 connection(s)
I20260809 02:05:10.107131 16149 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/ts-2/data/info.pb
I20260809 02:05:10.115110 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 16149
Aug 09 02:05:10 dist-test-slave-ss4n krb5kdc[14725](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241110, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.195@KRBTEST.COM for kudu/127.1.192.254@KRBTEST.COM
I20260809 02:05:10.124039 16005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:10.109290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.195:55381 (local address 127.1.192.254:37883)
0809 02:05:10.109459 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0809 02:05:10.109464 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:05:10.110397 (+ 933us) server_negotiation.cc:407] Connection header received
0809 02:05:10.111508 (+ 1111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:05:10.111512 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:05:10.111571 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:05:10.111719 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:05:10.112986 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:10.113836 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:10.115493 (+ 1657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:10.115836 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:10.115967 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:05:10.119315 (+ 3348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:05:10.119319 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:05:10.119322 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:05:10.119351 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:05:10.121402 (+ 2051us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:05:10.122096 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:05:10.122102 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:05:10.122107 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:05:10.122229 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:05:10.122634 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:05:10.122638 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:05:10.122640 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:05:10.122833 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:05:10.122948 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0809 02:05:10.123723 (+ 775us) server_negotiation.cc:299] Negotiation successful
0809 02:05:10.123852 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 02:05:10.126343 16295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:10.109707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:37883 (local address 127.1.192.195:55381)
0809 02:05:10.110204 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:10.110262 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0809 02:05:10.111208 (+ 946us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:05:10.111921 (+ 713us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:05:10.111928 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:05:10.112217 (+ 289us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:05:10.112799 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:05:10.112821 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:05:10.113983 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:05:10.113986 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:05:10.115294 (+ 1308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:05:10.115309 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:05:10.116160 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:05:10.116166 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:05:10.116383 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:05:10.117107 (+ 724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:05:10.117129 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:05:10.119130 (+ 2001us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:05:10.121546 (+ 2416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:05:10.121553 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:05:10.121571 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:05:10.121973 (+ 402us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:05:10.122334 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:05:10.122339 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:05:10.122341 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:05:10.122514 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:05:10.122948 (+ 434us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:05:10.122957 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:05:10.123318 (+ 361us) client_negotiation.cc:769] Sending connection context
0809 02:05:10.123915 (+ 597us) client_negotiation.cc:240] Negotiation successful
0809 02:05:10.124680 (+ 765us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":138,"threads_started":1}
I20260809 02:05:10.127444 16296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:10.115887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33310 (local address 127.1.192.254:37883)
0809 02:05:10.116230 (+ 343us) server_negotiation.cc:206] Beginning negotiation
0809 02:05:10.116237 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 02:05:10.116492 (+ 255us) server_negotiation.cc:407] Connection header received
0809 02:05:10.116677 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:05:10.116683 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:05:10.116740 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:05:10.116819 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:05:10.118814 (+ 1995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:10.119482 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:10.120937 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:10.121387 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:10.121550 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:05:10.123293 (+ 1743us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:05:10.123297 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:05:10.123300 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:05:10.123324 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:05:10.125199 (+ 1875us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:05:10.125743 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:05:10.125747 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:05:10.125748 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:05:10.125842 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:05:10.126193 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:05:10.126196 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:05:10.126198 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:05:10.126438 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:05:10.127030 (+ 592us) server_negotiation.cc:1035] Waiting for connection context
0809 02:05:10.127130 (+ 100us) server_negotiation.cc:299] Negotiation successful
0809 02:05:10.127275 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":267,"thread_start_us":133,"threads_started":1}
I20260809 02:05:10.127691 16293 heartbeater.cc:347] Connected to a master server at 127.1.192.254:37883
I20260809 02:05:10.127955 16293 heartbeater.cc:464] Registering TS with master...
I20260809 02:05:10.128587 16293 heartbeater.cc:511] Master 127.1.192.254:37883 requested a full tablet report, sending...
I20260809 02:05:10.130038 15787 ts_manager.cc:194] Registered new tserver with Master: df9f9d6c29894056bc556be4ff9992ba (127.1.192.195:34419)
I20260809 02:05:10.131438 15787 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.192.195@KRBTEST.COM'} at 127.1.192.195:55381
Aug 09 02:05:10 dist-test-slave-ss4n krb5kdc[14725](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786241110, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:05:10.586436 16003 heartbeater.cc:464] Registering TS with master...
I20260809 02:05:10.586772 16003 heartbeater.cc:511] Master 127.1.192.254:37883 requested a full tablet report, sending...
I20260809 02:05:10.587682 15787 ts_manager.cc:194] Registered new tserver with Master: c1654b360ae746338fafa519680cd83c (127.1.192.193:36407)
I20260809 02:05:10.588481 15787 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.192.193@KRBTEST.COM'} at 127.1.192.193:44947
Aug 09 02:05:10 dist-test-slave-ss4n krb5kdc[14725](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241110, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.254@KRBTEST.COM for HTTP/127.1.192.212@KRBTEST.COM
I20260809 02:05:10.871178 16147 heartbeater.cc:464] Registering TS with master...
I20260809 02:05:10.871523 16147 heartbeater.cc:511] Master 127.1.192.254:37883 requested a full tablet report, sending...
I20260809 02:05:10.872470 15787 ts_manager.cc:194] Registered new tserver with Master: 3ff9b9358080455d855a18628c1952fc (127.1.192.194:36917)
I20260809 02:05:10.873255 15787 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.192.194@KRBTEST.COM'} at 127.1.192.194:60485
I20260809 02:05:10.876510 1795 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 02:05:10.893666 16309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:10.878185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33324 (local address 127.1.192.254:37883)
0809 02:05:10.878431 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0809 02:05:10.878435 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:05:10.878576 (+ 141us) server_negotiation.cc:407] Connection header received
0809 02:05:10.878736 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:05:10.878739 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:05:10.879098 (+ 359us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:05:10.879196 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:05:10.880779 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:10.882294 (+ 1515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:10.883445 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:10.883782 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:10.883868 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:05:10.885821 (+ 1953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:05:10.885826 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:05:10.885830 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:05:10.885865 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:05:10.889301 (+ 3436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:05:10.890777 (+ 1476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:05:10.890781 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:05:10.890783 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:05:10.890890 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:05:10.891947 (+ 1057us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:05:10.891952 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:05:10.891954 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:05:10.892221 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:05:10.892361 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0809 02:05:10.893130 (+ 769us) server_negotiation.cc:299] Negotiation successful
0809 02:05:10.893440 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":78,"threads_started":1}
I20260809 02:05:10.900360 15787 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33324:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 02:05:10.905711 15787 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:05:10.944272 16315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:10.923852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.193:36407 (local address 127.0.0.1:55854)
0809 02:05:10.929268 (+ 5416us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:10.929285 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 02:05:10.929383 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:05:10.931212 (+ 1829us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:05:10.931215 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:05:10.931227 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:05:10.931289 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:05:10.931293 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:05:10.938411 (+ 7118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:05:10.938415 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:05:10.939541 (+ 1126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:05:10.939554 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:05:10.942683 (+ 3129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:05:10.942687 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:05:10.943796 (+ 1109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:05:10.943845 (+ 49us) client_negotiation.cc:769] Sending connection context
0809 02:05:10.943963 (+ 118us) client_negotiation.cc:240] Negotiation successful
0809 02:05:10.944109 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5333,"thread_start_us":94,"threads_started":1}
I20260809 02:05:10.946321 16320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:10.922620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.194:36917 (local address 127.0.0.1:60600)
0809 02:05:10.926747 (+ 4127us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:10.926761 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 02:05:10.926851 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:05:10.927875 (+ 1024us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:05:10.927882 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:05:10.927905 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:05:10.930890 (+ 2985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:05:10.930900 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:05:10.934440 (+ 3540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:05:10.934444 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:05:10.935892 (+ 1448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:05:10.935905 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:05:10.945882 (+ 9977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:05:10.945886 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:05:10.945989 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:05:10.946013 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 02:05:10.946094 (+ 81us) client_negotiation.cc:240] Negotiation successful
0809 02:05:10.946180 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4066,"thread_start_us":103,"threads_started":1}
I20260809 02:05:10.948002 16317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:10.919948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.195:34419 (local address 127.0.0.1:59716)
0809 02:05:10.926069 (+ 6121us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:10.926140 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0809 02:05:10.926275 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:05:10.931007 (+ 4732us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:05:10.931010 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:05:10.931027 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:05:10.931122 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:05:10.931127 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:05:10.940161 (+ 9034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:05:10.940165 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:05:10.941243 (+ 1078us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:05:10.941254 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:05:10.947606 (+ 6352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:05:10.947610 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:05:10.947703 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:05:10.947725 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 02:05:10.947793 (+ 68us) client_negotiation.cc:240] Negotiation successful
0809 02:05:10.947866 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5915,"thread_start_us":138,"threads_started":1}
I20260809 02:05:10.949332 16322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:10.924268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55854 (local address 127.1.192.193:36407)
0809 02:05:10.927088 (+ 2820us) server_negotiation.cc:206] Beginning negotiation
0809 02:05:10.927095 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 02:05:10.929518 (+ 2423us) server_negotiation.cc:407] Connection header received
0809 02:05:10.929617 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:05:10.929622 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:05:10.929842 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:05:10.929984 (+ 142us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:05:10.931406 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:10.938271 (+ 6865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:10.941409 (+ 3138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:10.942327 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:10.942434 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:05:10.942615 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0809 02:05:10.949044 (+ 6429us) server_negotiation.cc:299] Negotiation successful
0809 02:05:10.949186 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2714,"thread_start_us":79,"threads_started":1}
I20260809 02:05:10.951251 16319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:10.923037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60600 (local address 127.1.192.194:36917)
0809 02:05:10.926608 (+ 3571us) server_negotiation.cc:206] Beginning negotiation
0809 02:05:10.926617 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 02:05:10.927145 (+ 528us) server_negotiation.cc:407] Connection header received
0809 02:05:10.927233 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:05:10.927239 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:05:10.927502 (+ 263us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:05:10.927676 (+ 174us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:05:10.933148 (+ 5472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:10.934300 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:10.944654 (+ 10354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:10.945545 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:10.945665 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:05:10.945814 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0809 02:05:10.949640 (+ 3826us) server_negotiation.cc:299] Negotiation successful
0809 02:05:10.951110 (+ 1470us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3446,"thread_start_us":96,"threads_started":1}
I20260809 02:05:10.951037 16316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:10.920983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59716 (local address 127.1.192.195:34419)
0809 02:05:10.930286 (+ 9303us) server_negotiation.cc:206] Beginning negotiation
0809 02:05:10.930292 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:05:10.930316 (+ 24us) server_negotiation.cc:407] Connection header received
0809 02:05:10.930386 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:05:10.930392 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:05:10.930564 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:05:10.930731 (+ 167us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:05:10.936066 (+ 5335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:10.940027 (+ 3961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:10.946436 (+ 6409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:10.947282 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:10.947390 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:05:10.947531 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0809 02:05:10.950748 (+ 3217us) server_negotiation.cc:299] Negotiation successful
0809 02:05:10.950883 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9192,"thread_start_us":81,"threads_started":1}
I20260809 02:05:10.954407 15929 tablet_service.cc:1467] Processing CreateTablet for tablet 0a1e7f6c756248dabf769289808d308d (DEFAULT_TABLE table=test-table [id=33f08df49bcc4ad7a6a930a5c7bbddc5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:05:10.954407 16075 tablet_service.cc:1467] Processing CreateTablet for tablet 0a1e7f6c756248dabf769289808d308d (DEFAULT_TABLE table=test-table [id=33f08df49bcc4ad7a6a930a5c7bbddc5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:05:10.954627 16223 tablet_service.cc:1467] Processing CreateTablet for tablet 0a1e7f6c756248dabf769289808d308d (DEFAULT_TABLE table=test-table [id=33f08df49bcc4ad7a6a930a5c7bbddc5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:05:10.955612 16075 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a1e7f6c756248dabf769289808d308d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:05:10.955621 15929 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a1e7f6c756248dabf769289808d308d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:05:10.955662 16223 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a1e7f6c756248dabf769289808d308d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:05:10.965536 16325 tablet_bootstrap.cc:492] T 0a1e7f6c756248dabf769289808d308d P df9f9d6c29894056bc556be4ff9992ba: Bootstrap starting.
I20260809 02:05:10.965830 16323 tablet_bootstrap.cc:492] T 0a1e7f6c756248dabf769289808d308d P 3ff9b9358080455d855a18628c1952fc: Bootstrap starting.
I20260809 02:05:10.967811 16325 tablet_bootstrap.cc:654] T 0a1e7f6c756248dabf769289808d308d P df9f9d6c29894056bc556be4ff9992ba: Neither blocks nor log segments found. Creating new log.
I20260809 02:05:10.967979 16323 tablet_bootstrap.cc:654] T 0a1e7f6c756248dabf769289808d308d P 3ff9b9358080455d855a18628c1952fc: Neither blocks nor log segments found. Creating new log.
I20260809 02:05:10.968626 16325 log.cc:824] T 0a1e7f6c756248dabf769289808d308d P df9f9d6c29894056bc556be4ff9992ba: Log is configured to *not* fsync() on all Append() calls
I20260809 02:05:10.968791 16323 log.cc:824] T 0a1e7f6c756248dabf769289808d308d P 3ff9b9358080455d855a18628c1952fc: Log is configured to *not* fsync() on all Append() calls
I20260809 02:05:10.969717 16324 tablet_bootstrap.cc:492] T 0a1e7f6c756248dabf769289808d308d P c1654b360ae746338fafa519680cd83c: Bootstrap starting.
I20260809 02:05:10.971028 16325 tablet_bootstrap.cc:492] T 0a1e7f6c756248dabf769289808d308d P df9f9d6c29894056bc556be4ff9992ba: No bootstrap required, opened a new log
I20260809 02:05:10.971244 16325 ts_tablet_manager.cc:1397] T 0a1e7f6c756248dabf769289808d308d P df9f9d6c29894056bc556be4ff9992ba: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260809 02:05:10.971606 16323 tablet_bootstrap.cc:492] T 0a1e7f6c756248dabf769289808d308d P 3ff9b9358080455d855a18628c1952fc: No bootstrap required, opened a new log
I20260809 02:05:10.971769 16323 ts_tablet_manager.cc:1397] T 0a1e7f6c756248dabf769289808d308d P 3ff9b9358080455d855a18628c1952fc: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260809 02:05:10.971920 16324 tablet_bootstrap.cc:654] T 0a1e7f6c756248dabf769289808d308d P c1654b360ae746338fafa519680cd83c: Neither blocks nor log segments found. Creating new log.
I20260809 02:05:10.972728 16324 log.cc:824] T 0a1e7f6c756248dabf769289808d308d P c1654b360ae746338fafa519680cd83c: Log is configured to *not* fsync() on all Append() calls
I20260809 02:05:10.974972 16324 tablet_bootstrap.cc:492] T 0a1e7f6c756248dabf769289808d308d P c1654b360ae746338fafa519680cd83c: No bootstrap required, opened a new log
I20260809 02:05:10.975152 16324 ts_tablet_manager.cc:1397] T 0a1e7f6c756248dabf769289808d308d P c1654b360ae746338fafa519680cd83c: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260809 02:05:10.976075 16323 raft_consensus.cc:348] T 0a1e7f6c756248dabf769289808d308d P 3ff9b9358080455d855a18628c1952fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df9f9d6c29894056bc556be4ff9992ba" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 34419 } } peers { permanent_uuid: "3ff9b9358080455d855a18628c1952fc" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 36917 } } peers { permanent_uuid: "c1654b360ae746338fafa519680cd83c" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 36407 } }
I20260809 02:05:10.976352 16323 raft_consensus.cc:374] T 0a1e7f6c756248dabf769289808d308d P 3ff9b9358080455d855a18628c1952fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:05:10.976404 16323 raft_consensus.cc:729] T 0a1e7f6c756248dabf769289808d308d P 3ff9b9358080455d855a18628c1952fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ff9b9358080455d855a18628c1952fc, State: Initialized, Role: FOLLOWER
I20260809 02:05:10.976040 16325 raft_consensus.cc:348] T 0a1e7f6c756248dabf769289808d308d P df9f9d6c29894056bc556be4ff9992ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df9f9d6c29894056bc556be4ff9992ba" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 34419 } } peers { permanent_uuid: "3ff9b9358080455d855a18628c1952fc" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 36917 } } peers { permanent_uuid: "c1654b360ae746338fafa519680cd83c" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 36407 } }
I20260809 02:05:10.976624 16325 raft_consensus.cc:374] T 0a1e7f6c756248dabf769289808d308d P df9f9d6c29894056bc556be4ff9992ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:05:10.976680 16325 raft_consensus.cc:729] T 0a1e7f6c756248dabf769289808d308d P df9f9d6c29894056bc556be4ff9992ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df9f9d6c29894056bc556be4ff9992ba, State: Initialized, Role: FOLLOWER
I20260809 02:05:10.977082 16325 consensus_queue.cc:260] T 0a1e7f6c756248dabf769289808d308d P df9f9d6c29894056bc556be4ff9992ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df9f9d6c29894056bc556be4ff9992ba" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 34419 } } peers { permanent_uuid: "3ff9b9358080455d855a18628c1952fc" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 36917 } } peers { permanent_uuid: "c1654b360ae746338fafa519680cd83c" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 36407 } }
I20260809 02:05:10.977006 16323 consensus_queue.cc:260] T 0a1e7f6c756248dabf769289808d308d P 3ff9b9358080455d855a18628c1952fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df9f9d6c29894056bc556be4ff9992ba" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 34419 } } peers { permanent_uuid: "3ff9b9358080455d855a18628c1952fc" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 36917 } } peers { permanent_uuid: "c1654b360ae746338fafa519680cd83c" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 36407 } }
I20260809 02:05:10.978197 16325 ts_tablet_manager.cc:1428] T 0a1e7f6c756248dabf769289808d308d P df9f9d6c29894056bc556be4ff9992ba: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260809 02:05:10.978197 16323 ts_tablet_manager.cc:1428] T 0a1e7f6c756248dabf769289808d308d P 3ff9b9358080455d855a18628c1952fc: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260809 02:05:10.978574 16293 heartbeater.cc:503] Master 127.1.192.254:37883 was elected leader, sending a full tablet report...
I20260809 02:05:10.978578 16147 heartbeater.cc:503] Master 127.1.192.254:37883 was elected leader, sending a full tablet report...
I20260809 02:05:10.979532 16324 raft_consensus.cc:348] T 0a1e7f6c756248dabf769289808d308d P c1654b360ae746338fafa519680cd83c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df9f9d6c29894056bc556be4ff9992ba" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 34419 } } peers { permanent_uuid: "3ff9b9358080455d855a18628c1952fc" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 36917 } } peers { permanent_uuid: "c1654b360ae746338fafa519680cd83c" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 36407 } }
I20260809 02:05:10.979859 16324 raft_consensus.cc:374] T 0a1e7f6c756248dabf769289808d308d P c1654b360ae746338fafa519680cd83c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:05:10.979938 16324 raft_consensus.cc:729] T 0a1e7f6c756248dabf769289808d308d P c1654b360ae746338fafa519680cd83c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1654b360ae746338fafa519680cd83c, State: Initialized, Role: FOLLOWER
I20260809 02:05:10.980389 16324 consensus_queue.cc:260] T 0a1e7f6c756248dabf769289808d308d P c1654b360ae746338fafa519680cd83c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df9f9d6c29894056bc556be4ff9992ba" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 34419 } } peers { permanent_uuid: "3ff9b9358080455d855a18628c1952fc" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 36917 } } peers { permanent_uuid: "c1654b360ae746338fafa519680cd83c" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 36407 } }
I20260809 02:05:10.981040 16003 heartbeater.cc:503] Master 127.1.192.254:37883 was elected leader, sending a full tablet report...
I20260809 02:05:10.981431 16324 ts_tablet_manager.cc:1428] T 0a1e7f6c756248dabf769289808d308d P c1654b360ae746338fafa519680cd83c: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
W20260809 02:05:11.044296 16004 tablet.cc:2369] T 0a1e7f6c756248dabf769289808d308d P c1654b360ae746338fafa519680cd83c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 02:05:11.076247 16148 tablet.cc:2369] T 0a1e7f6c756248dabf769289808d308d P 3ff9b9358080455d855a18628c1952fc: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 02:05:11.108888 16294 tablet.cc:2369] T 0a1e7f6c756248dabf769289808d308d P df9f9d6c29894056bc556be4ff9992ba: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 02:05:11.115698 16330 raft_consensus.cc:482] T 0a1e7f6c756248dabf769289808d308d P 3ff9b9358080455d855a18628c1952fc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:05:11.115942 16330 raft_consensus.cc:504] T 0a1e7f6c756248dabf769289808d308d P 3ff9b9358080455d855a18628c1952fc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df9f9d6c29894056bc556be4ff9992ba" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 34419 } } peers { permanent_uuid: "3ff9b9358080455d855a18628c1952fc" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 36917 } } peers { permanent_uuid: "c1654b360ae746338fafa519680cd83c" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 36407 } }
I20260809 02:05:11.117367 16330 leader_election.cc:288] T 0a1e7f6c756248dabf769289808d308d P 3ff9b9358080455d855a18628c1952fc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers df9f9d6c29894056bc556be4ff9992ba (127.1.192.195:34419), c1654b360ae746338fafa519680cd83c (127.1.192.193:36407)
I20260809 02:05:11.127012 16316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:11.121058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.194:35875 (local address 127.1.192.195:34419)
0809 02:05:11.121209 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0809 02:05:11.121215 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:05:11.122385 (+ 1170us) server_negotiation.cc:407] Connection header received
0809 02:05:11.122627 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:05:11.122631 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:05:11.122684 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:05:11.122735 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:05:11.123328 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:11.124140 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:11.125721 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:11.126266 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:11.126364 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:05:11.126419 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 02:05:11.126776 (+ 357us) server_negotiation.cc:299] Negotiation successful
0809 02:05:11.126860 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 02:05:11.127640 16335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:11.121007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.195:34419 (local address 127.1.192.194:35875)
0809 02:05:11.122271 (+ 1264us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:11.122291 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 02:05:11.122479 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:05:11.122919 (+ 440us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:05:11.122922 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:05:11.122943 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:05:11.123183 (+ 240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:05:11.123191 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:05:11.124340 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:05:11.124343 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:05:11.125578 (+ 1235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:05:11.125589 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:05:11.126525 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:05:11.126529 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:05:11.126633 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:05:11.126656 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 02:05:11.127404 (+ 748us) client_negotiation.cc:240] Negotiation successful
0809 02:05:11.127493 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1074,"thread_start_us":160,"threads_started":1}
I20260809 02:05:11.128538 16244 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a1e7f6c756248dabf769289808d308d" candidate_uuid: "3ff9b9358080455d855a18628c1952fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df9f9d6c29894056bc556be4ff9992ba" is_pre_election: true
I20260809 02:05:11.129009 16244 raft_consensus.cc:2454] T 0a1e7f6c756248dabf769289808d308d P df9f9d6c29894056bc556be4ff9992ba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ff9b9358080455d855a18628c1952fc in term 0.
I20260809 02:05:11.129796 16026 leader_election.cc:302] T 0a1e7f6c756248dabf769289808d308d P 3ff9b9358080455d855a18628c1952fc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ff9b9358080455d855a18628c1952fc, df9f9d6c29894056bc556be4ff9992ba; no voters:
I20260809 02:05:11.130153 16330 raft_consensus.cc:2781] T 0a1e7f6c756248dabf769289808d308d P 3ff9b9358080455d855a18628c1952fc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:05:11.130266 16330 raft_consensus.cc:482] T 0a1e7f6c756248dabf769289808d308d P 3ff9b9358080455d855a18628c1952fc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:05:11.130314 16330 raft_consensus.cc:3037] T 0a1e7f6c756248dabf769289808d308d P 3ff9b9358080455d855a18628c1952fc [term 0 FOLLOWER]: Advancing to term 1
Aug 09 02:05:11 dist-test-slave-ss4n krb5kdc[14725](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241110, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.254@KRBTEST.COM for HTTP/127.1.192.212@KRBTEST.COM
I20260809 02:05:11.131757 16330 raft_consensus.cc:504] T 0a1e7f6c756248dabf769289808d308d P 3ff9b9358080455d855a18628c1952fc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df9f9d6c29894056bc556be4ff9992ba" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 34419 } } peers { permanent_uuid: "3ff9b9358080455d855a18628c1952fc" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 36917 } } peers { permanent_uuid: "c1654b360ae746338fafa519680cd83c" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 36407 } }
I20260809 02:05:11.132679 16244 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a1e7f6c756248dabf769289808d308d" candidate_uuid: "3ff9b9358080455d855a18628c1952fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df9f9d6c29894056bc556be4ff9992ba"
I20260809 02:05:11.132818 16244 raft_consensus.cc:3037] T 0a1e7f6c756248dabf769289808d308d P df9f9d6c29894056bc556be4ff9992ba [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:05:11.133502 16322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:11.121952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.194:38051 (local address 127.1.192.193:36407)
0809 02:05:11.122102 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0809 02:05:11.122107 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:05:11.128201 (+ 6094us) server_negotiation.cc:407] Connection header received
0809 02:05:11.128401 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:05:11.128405 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:05:11.128457 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:05:11.128607 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:05:11.129131 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:11.129922 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:11.131925 (+ 2003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:11.132729 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:11.132871 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:05:11.132927 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 02:05:11.133264 (+ 337us) server_negotiation.cc:299] Negotiation successful
0809 02:05:11.133351 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 02:05:11.133910 16335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:11.121521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.193:36407 (local address 127.1.192.194:38051)
0809 02:05:11.128122 (+ 6601us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:11.128137 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 02:05:11.128280 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:05:11.128823 (+ 543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:05:11.128826 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:05:11.128860 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:05:11.128940 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:05:11.128946 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:05:11.130835 (+ 1889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:05:11.130839 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:05:11.131765 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:05:11.131775 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:05:11.133008 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:05:11.133012 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:05:11.133113 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:05:11.133137 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 02:05:11.133706 (+ 569us) client_negotiation.cc:240] Negotiation successful
0809 02:05:11.133787 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6544,"thread_start_us":89,"threads_started":1}
I20260809 02:05:11.134138 16244 raft_consensus.cc:2454] T 0a1e7f6c756248dabf769289808d308d P df9f9d6c29894056bc556be4ff9992ba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ff9b9358080455d855a18628c1952fc in term 1.
I20260809 02:05:11.134894 16026 leader_election.cc:302] T 0a1e7f6c756248dabf769289808d308d P 3ff9b9358080455d855a18628c1952fc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ff9b9358080455d855a18628c1952fc, df9f9d6c29894056bc556be4ff9992ba; no voters:
I20260809 02:05:11.134910 15951 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a1e7f6c756248dabf769289808d308d" candidate_uuid: "3ff9b9358080455d855a18628c1952fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1654b360ae746338fafa519680cd83c" is_pre_election: true
I20260809 02:05:11.135238 15951 raft_consensus.cc:2454] T 0a1e7f6c756248dabf769289808d308d P c1654b360ae746338fafa519680cd83c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ff9b9358080455d855a18628c1952fc in term 0.
I20260809 02:05:11.135389 16330 leader_election.cc:288] T 0a1e7f6c756248dabf769289808d308d P 3ff9b9358080455d855a18628c1952fc [CANDIDATE]: Term 1 election: Requested vote from peers df9f9d6c29894056bc556be4ff9992ba (127.1.192.195:34419), c1654b360ae746338fafa519680cd83c (127.1.192.193:36407)
I20260809 02:05:11.135506 16330 raft_consensus.cc:2781] T 0a1e7f6c756248dabf769289808d308d P 3ff9b9358080455d855a18628c1952fc [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:05:11.135582 16330 raft_consensus.cc:686] T 0a1e7f6c756248dabf769289808d308d P 3ff9b9358080455d855a18628c1952fc [term 1 LEADER]: Becoming Leader. State: Replica: 3ff9b9358080455d855a18628c1952fc, State: Running, Role: LEADER
I20260809 02:05:11.135574 15950 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a1e7f6c756248dabf769289808d308d" candidate_uuid: "3ff9b9358080455d855a18628c1952fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1654b360ae746338fafa519680cd83c"
I20260809 02:05:11.135684 15950 raft_consensus.cc:3037] T 0a1e7f6c756248dabf769289808d308d P c1654b360ae746338fafa519680cd83c [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:05:11.135950 16330 consensus_queue.cc:237] T 0a1e7f6c756248dabf769289808d308d P 3ff9b9358080455d855a18628c1952fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df9f9d6c29894056bc556be4ff9992ba" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 34419 } } peers { permanent_uuid: "3ff9b9358080455d855a18628c1952fc" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 36917 } } peers { permanent_uuid: "c1654b360ae746338fafa519680cd83c" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 36407 } }
I20260809 02:05:11.136956 15950 raft_consensus.cc:2454] T 0a1e7f6c756248dabf769289808d308d P c1654b360ae746338fafa519680cd83c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ff9b9358080455d855a18628c1952fc in term 1.
I20260809 02:05:11.143069 15785 catalog_manager.cc:5495] T 0a1e7f6c756248dabf769289808d308d P 3ff9b9358080455d855a18628c1952fc reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ff9b9358080455d855a18628c1952fc (127.1.192.194). New cstate: current_term: 1 leader_uuid: "3ff9b9358080455d855a18628c1952fc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df9f9d6c29894056bc556be4ff9992ba" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 34419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ff9b9358080455d855a18628c1952fc" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 36917 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1654b360ae746338fafa519680cd83c" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 36407 } health_report { overall_health: UNKNOWN } } }
I20260809 02:05:11.167316 16319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:11.162051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60606 (local address 127.1.192.194:36917)
0809 02:05:11.162187 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0809 02:05:11.162192 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:05:11.162208 (+ 16us) server_negotiation.cc:407] Connection header received
0809 02:05:11.162306 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:05:11.162309 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:05:11.162373 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:05:11.162518 (+ 145us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:05:11.162898 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:11.163741 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:11.165080 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:11.165431 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:11.165613 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:05:11.166028 (+ 415us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:05:11.166599 (+ 571us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:05:11.166716 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0809 02:05:11.167095 (+ 379us) server_negotiation.cc:299] Negotiation successful
0809 02:05:11.167160 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 02:05:11.189858 15950 raft_consensus.cc:1261] T 0a1e7f6c756248dabf769289808d308d P c1654b360ae746338fafa519680cd83c [term 1 FOLLOWER]: Refusing update from remote peer 3ff9b9358080455d855a18628c1952fc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:05:11.190294 16244 raft_consensus.cc:1261] T 0a1e7f6c756248dabf769289808d308d P df9f9d6c29894056bc556be4ff9992ba [term 1 FOLLOWER]: Refusing update from remote peer 3ff9b9358080455d855a18628c1952fc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:05:11.190713 16330 consensus_queue.cc:1035] T 0a1e7f6c756248dabf769289808d308d P 3ff9b9358080455d855a18628c1952fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1654b360ae746338fafa519680cd83c" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 36407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:05:11.191035 16336 consensus_queue.cc:1035] T 0a1e7f6c756248dabf769289808d308d P 3ff9b9358080455d855a18628c1952fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "df9f9d6c29894056bc556be4ff9992ba" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 34419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:05:11.203951 16341 mvcc.cc:205] Tried to move back new op lower bound from 7316443591421722624 to 7316443591216537600. Current Snapshot: MvccSnapshot[applied={T|T < 7316443591421722624}]
I20260809 02:05:11.207975 16342 mvcc.cc:205] Tried to move back new op lower bound from 7316443591421722624 to 7316443591216537600. Current Snapshot: MvccSnapshot[applied={T|T < 7316443591421722624}]
I20260809 02:05:11.219631 15787 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33324:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:05:11.219849 15787 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33324:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:05:11.222841 15787 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c08cd059ce0e458ca8d1f1a00f024e0c: Sending DeleteTablet for 3 replicas of tablet 0a1e7f6c756248dabf769289808d308d
I20260809 02:05:11.223835 16223 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a1e7f6c756248dabf769289808d308d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:05:11 UTC) from {username='kudu', principal='kudu/127.1.192.254@KRBTEST.COM'} at 127.0.0.1:59716
I20260809 02:05:11.225238 16075 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a1e7f6c756248dabf769289808d308d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:05:11 UTC) from {username='kudu', principal='kudu/127.1.192.254@KRBTEST.COM'} at 127.0.0.1:60600
I20260809 02:05:11.225845 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 15828
I20260809 02:05:11.225960 15929 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a1e7f6c756248dabf769289808d308d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:05:11 UTC) from {username='kudu', principal='kudu/127.1.192.254@KRBTEST.COM'} at 127.0.0.1:55854
I20260809 02:05:11.234377 16351 tablet_replica.cc:330] T 0a1e7f6c756248dabf769289808d308d P 3ff9b9358080455d855a18628c1952fc: stopping tablet replica
I20260809 02:05:11.234413 16350 tablet_replica.cc:330] T 0a1e7f6c756248dabf769289808d308d P df9f9d6c29894056bc556be4ff9992ba: stopping tablet replica
I20260809 02:05:11.234828 16350 raft_consensus.cc:2229] T 0a1e7f6c756248dabf769289808d308d P df9f9d6c29894056bc556be4ff9992ba [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:05:11.234861 16351 raft_consensus.cc:2229] T 0a1e7f6c756248dabf769289808d308d P 3ff9b9358080455d855a18628c1952fc [term 1 LEADER]: Raft consensus shutting down.
I20260809 02:05:11.235334 16350 raft_consensus.cc:2258] T 0a1e7f6c756248dabf769289808d308d P df9f9d6c29894056bc556be4ff9992ba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:05:11.235399 16351 raft_consensus.cc:2258] T 0a1e7f6c756248dabf769289808d308d P 3ff9b9358080455d855a18628c1952fc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:05:11.237152 16351 ts_tablet_manager.cc:1905] T 0a1e7f6c756248dabf769289808d308d P 3ff9b9358080455d855a18628c1952fc: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 02:05:11.237960 15774 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.192.193:36407 (error 108)
I20260809 02:05:11.238891 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 16007
I20260809 02:05:11.243964 16315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:11.243632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.193:36407 (local address 127.0.0.1:55860)
0809 02:05:11.243779 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:11.243868 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.193:36407: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260809 02:05:11.244346 15774 catalog_manager.cc:4584] TS c1654b360ae746338fafa519680cd83c (127.1.192.193:36407): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0a1e7f6c756248dabf769289808d308d: Network error: Client connection negotiation failed: client connection to 127.1.192.193:36407: connect: Connection refused (error 111)
I20260809 02:05:11.244930 16350 ts_tablet_manager.cc:1905] T 0a1e7f6c756248dabf769289808d308d P df9f9d6c29894056bc556be4ff9992ba: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:05:11.248631 16350 ts_tablet_manager.cc:1918] T 0a1e7f6c756248dabf769289808d308d P df9f9d6c29894056bc556be4ff9992ba: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:05:11.248880 16350 log.cc:1196] T 0a1e7f6c756248dabf769289808d308d P df9f9d6c29894056bc556be4ff9992ba: Deleting WAL directory at /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786240941968412-1795-0/minicluster-data/ts-2/wal/wals/0a1e7f6c756248dabf769289808d308d
I20260809 02:05:11.249493 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 16149
I20260809 02:05:11.249714 16315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:11.249342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.194:36917 (local address 127.0.0.1:60610)
0809 02:05:11.249546 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:11.249613 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.194:36917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
I20260809 02:05:11.257786 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 15751
I20260809 02:05:11.267022 1795 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260809 02:05:11.613000 1795 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260809 02:05:11.613171 1795 mini_ranger.cc:66] Stopping Ranger...
I20260809 02:05:11.959235 1795 mini_ranger.cc:68] Stopped Ranger
2026-08-09 02:05:11.959 UTC [14756] LOG: received smart shutdown request
2026-08-09 02:05:11.964 UTC [14756] LOG: background worker "logical replication launcher" (PID 14765) exited with exit code 1
2026-08-09 02:05:11.964 UTC [14760] LOG: shutting down
2026-08-09 02:05:12.157 UTC [14756] LOG: database system is shut down
2026-08-09T02:05:12Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (52865 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786240941968412-1795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:05:12 dist-test-slave-ss4n krb5kdc[16361](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:05:12 dist-test-slave-ss4n krb5kdc[16361](info): set up 2 sockets
Aug 09 02:05:12 dist-test-slave-ss4n krb5kdc[16361](info): commencing operation
krb5kdc: starting...
W20260809 02:05:13.408291 1795 mini_kdc.cc:121] Time spent starting KDC: real 1.086s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786240941968412-1795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786240941968412-1795-0/krb5kdc/test-user.keytab.
Aug 09 02:05:13 dist-test-slave-ss4n krb5kdc[16361](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241113, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:05:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:05:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.192.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.192.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.192.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab.
Entry for principal kudu/127.1.192.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab.
I20260809 02:05:13.553721 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786240941968412-1795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786240941968412-1795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786240941968412-1795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.192.254:43569
--webserver_interface=127.1.192.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab
--principal=kudu/127.1.192.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:33853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.192.254:43569,127.1.192.253:39993,127.1.192.252:46255
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:05:13.670648 16374 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:05:13.671020 16374 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:05:13.671116 16374 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:05:13.675719 16374 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:05:13.675834 16374 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:05:13.675879 16374 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:05:13.675926 16374 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:05:13.675959 16374 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:05:13.682219 16374 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:33853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786240941968412-1795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786240941968412-1795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.192.254:43569,127.1.192.253:39993,127.1.192.252:46255
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab
--principal=kudu/127.1.192.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.192.254:43569
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.192.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16374
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786240941968412-1795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:05:13.683447 16374 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:05:13.684562 16374 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:05:13.693894 16380 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:05:13.693902 16379 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:05:13.693894 16382 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:05:13.694335 16374 server_base.cc:1034] running on GCE node
I20260809 02:05:13.695053 16374 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:05:13.696316 16374 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:05:13.697619 16374 hybrid_clock.cc:648] HybridClock initialized: now 1786241113697599 us; error 33 us; skew 500 ppm
Aug 09 02:05:13 dist-test-slave-ss4n krb5kdc[16361](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241113, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:05:13.700743 16374 init.cc:377] Logged in from keytab as kudu/127.1.192.254@KRBTEST.COM (short username kudu)
I20260809 02:05:13.701809 16374 webserver.cc:460] Webserver started at http://127.1.192.254:39499/ using document root <none> and password file <none>
I20260809 02:05:13.702438 16374 fs_manager.cc:359] Metadata directory not provided
I20260809 02:05:13.702533 16374 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:05:13.702762 16374 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:05:13.704824 16374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786240941968412-1795-0/minicluster-data/master-0/data/instance:
uuid: "9815451a73d548b8acc0309c51c4a3ca"
format_stamp: "Formatted at 2026-08-09 02:05:13 on dist-test-slave-ss4n"
server_key: "0de79d1b1899ae2f298a90920fcee2fb"
server_key_iv: "c6ee52e02f29c5f2e0e4a11c651b5cb5"
server_key_version: "encryptionkey@0"
I20260809 02:05:13.705395 16374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786240941968412-1795-0/minicluster-data/master-0/wal/instance:
uuid: "9815451a73d548b8acc0309c51c4a3ca"
format_stamp: "Formatted at 2026-08-09 02:05:13 on dist-test-slave-ss4n"
server_key: "0de79d1b1899ae2f298a90920fcee2fb"
server_key_iv: "c6ee52e02f29c5f2e0e4a11c651b5cb5"
server_key_version: "encryptionkey@0"
I20260809 02:05:13.709163 16374 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260809 02:05:13.711896 16389 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:05:13.713240 16374 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 02:05:13.713429 16374 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786240941968412-1795-0/minicluster-data/master-0/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786240941968412-1795-0/minicluster-data/master-0/wal
uuid: "9815451a73d548b8acc0309c51c4a3ca"
format_stamp: "Formatted at 2026-08-09 02:05:13 on dist-test-slave-ss4n"
server_key: "0de79d1b1899ae2f298a90920fcee2fb"
server_key_iv: "c6ee52e02f29c5f2e0e4a11c651b5cb5"
server_key_version: "encryptionkey@0"
I20260809 02:05:13.713624 16374 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786240941968412-1795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786240941968412-1795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786240941968412-1795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:05:13.737730 16374 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:05:13.741571 16374 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:05:13.741828 16374 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:05:13.750269 16374 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.254:43569
I20260809 02:05:13.750301 16445 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.254:43569 every 8 connection(s)
I20260809 02:05:13.751390 16374 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
I20260809 02:05:13.751995 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 16374
I20260809 02:05:13.752157 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786240941968412-1795-0/minicluster-data/master-0/wal/instance
I20260809 02:05:13.752442 1795 external_mini_cluster.cc:1428] Setting key 27cdb73132b3840503a0bab825e4c8d1
I20260809 02:05:13.754612 16446 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:05:13.758929 16446 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.192.254" port: 43569 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.1.192.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.192.253@KRBTEST.COM" created.
Aug 09 02:05:13 dist-test-slave-ss4n krb5kdc[16361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241113, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.254@KRBTEST.COM for kudu/127.1.192.254@KRBTEST.COM
I20260809 02:05:13.776587 16451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:13.761544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:43569 (local address 127.0.0.1:36046)
0809 02:05:13.762136 (+ 592us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:13.762202 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0809 02:05:13.763179 (+ 977us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:05:13.764040 (+ 861us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:05:13.764045 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:05:13.764552 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:05:13.764950 (+ 398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:05:13.764974 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:05:13.766351 (+ 1377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:05:13.766354 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:05:13.767145 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:05:13.767154 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:05:13.768064 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:05:13.768067 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:05:13.768183 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:05:13.768924 (+ 741us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:05:13.768942 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:05:13.770842 (+ 1900us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:05:13.773349 (+ 2507us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:05:13.773358 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:05:13.773367 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:05:13.773724 (+ 357us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:05:13.774128 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:05:13.774130 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:05:13.774132 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:05:13.774252 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:05:13.774907 (+ 655us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:05:13.774912 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:05:13.775062 (+ 150us) client_negotiation.cc:769] Sending connection context
0809 02:05:13.775310 (+ 248us) client_negotiation.cc:240] Negotiation successful
0809 02:05:13.775624 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":192,"threads_started":1}
I20260809 02:05:13.776641 16449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:13.761544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36046 (local address 127.1.192.254:43569)
0809 02:05:13.762005 (+ 461us) server_negotiation.cc:206] Beginning negotiation
0809 02:05:13.762020 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 02:05:13.762315 (+ 295us) server_negotiation.cc:407] Connection header received
0809 02:05:13.763524 (+ 1209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:05:13.763529 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:05:13.763792 (+ 263us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:05:13.763993 (+ 201us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:05:13.765160 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:13.766222 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:13.767265 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:13.767880 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:13.768064 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:05:13.771013 (+ 2949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:05:13.771022 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:05:13.771038 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:05:13.771069 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:05:13.773213 (+ 2144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:05:13.773867 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:05:13.773872 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:05:13.773874 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:05:13.773989 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:05:13.774373 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:05:13.774376 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:05:13.774377 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:05:13.774777 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:05:13.774904 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0809 02:05:13.775450 (+ 546us) server_negotiation.cc:299] Negotiation successful
0809 02:05:13.775699 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":84,"server-negotiator.queue_time_us":302,"thread_start_us":146,"threads_started":1}
I20260809 02:05:13.779160 16446 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.192.253" port: 39993 } has no permanent_uuid. Determining permanent_uuid...
I20260809 02:05:13.779894 16451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:13.779625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.253:39993 (local address 127.0.0.1:33602)
0809 02:05:13.779751 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:13.779831 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.253:39993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260809 02:05:13.780272 16393 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.192.253:39993: connect: Connection refused (error 111)
I20260809 02:05:13.781311 16451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:13.781013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.253:39993 (local address 127.0.0.1:33614)
0809 02:05:13.781174 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:13.781240 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.253:39993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260809 02:05:13.781618 16446 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.192.253:39993: Network error: Client connection negotiation failed: client connection to 127.1.192.253:39993: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.192.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.253.keytab.
Entry for principal kudu/127.1.192.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.253.keytab.
I20260809 02:05:13.784827 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786240941968412-1795-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786240941968412-1795-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786240941968412-1795-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786240941968412-1795-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.192.253:39993
--webserver_interface=127.1.192.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.253.keytab
--principal=kudu/127.1.192.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:33853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.192.254:43569,127.1.192.253:39993,127.1.192.252:46255
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 02:05:13.830883 16446 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.192.253" port: 39993 } attempt: 1
I20260809 02:05:13.831936 16451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:13.831568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.253:39993 (local address 127.0.0.1:33628)
0809 02:05:13.831774 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:13.831846 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.253:39993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260809 02:05:13.832815 16451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:13.832562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.253:39993 (local address 127.0.0.1:33640)
0809 02:05:13.832691 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:13.832748 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.253:39993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260809 02:05:13.832971 16446 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.192.253:39993: Network error: Client connection negotiation failed: client connection to 127.1.192.253:39993: connect: Connection refused (error 111)
W20260809 02:05:13.900691 16454 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:05:13.901021 16454 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:05:13.901091 16454 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260809 02:05:13.901156 16446 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.192.253" port: 39993 } attempt: 2
I20260809 02:05:13.902050 16451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:13.901717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.253:39993 (local address 127.0.0.1:33654)
0809 02:05:13.901874 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:13.901946 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.253:39993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260809 02:05:13.903026 16451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:13.902758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.253:39993 (local address 127.0.0.1:33664)
0809 02:05:13.902902 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:13.902962 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.253:39993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260809 02:05:13.903200 16446 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.192.253:39993: Network error: Client connection negotiation failed: client connection to 127.1.192.253:39993: connect: Connection refused (error 111)
W20260809 02:05:13.905859 16454 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:05:13.905918 16454 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:05:13.905939 16454 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:05:13.905956 16454 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:05:13.906010 16454 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:05:13.911737 16454 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:33853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786240941968412-1795-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786240941968412-1795-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.1.192.254:43569,127.1.192.253:39993,127.1.192.252:46255
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.253.keytab
--principal=kudu/127.1.192.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.192.253:39993
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786240941968412-1795-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.1.192.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16454
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786240941968412-1795-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:05:13.912894 16454 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:05:13.914215 16454 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:05:13.925803 16460 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:05:13.925796 16459 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:05:13.925808 16462 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:05:13.926183 16454 server_base.cc:1034] running on GCE node
I20260809 02:05:13.926807 16454 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:05:13.927959 16454 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:05:13.929183 16454 hybrid_clock.cc:648] HybridClock initialized: now 1786241113929164 us; error 33 us; skew 500 ppm
Aug 09 02:05:13 dist-test-slave-ss4n krb5kdc[16361](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241113, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:05:13.932468 16454 init.cc:377] Logged in from keytab as kudu/127.1.192.253@KRBTEST.COM (short username kudu)
I20260809 02:05:13.933409 16454 webserver.cc:460] Webserver started at http://127.1.192.253:36547/ using document root <none> and password file <none>
I20260809 02:05:13.933993 16454 fs_manager.cc:359] Metadata directory not provided
I20260809 02:05:13.934091 16454 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:05:13.934319 16454 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:05:13.936375 16454 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786240941968412-1795-0/minicluster-data/master-1/data/instance:
uuid: "82734084774e4dc18dcc948aecea3d17"
format_stamp: "Formatted at 2026-08-09 02:05:13 on dist-test-slave-ss4n"
server_key: "a3e0354dd9e3c45d933b0eab315b4c1e"
server_key_iv: "965f5f445a01f2f7611ff6cb9731a7fb"
server_key_version: "encryptionkey@0"
I20260809 02:05:13.936952 16454 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786240941968412-1795-0/minicluster-data/master-1/wal/instance:
uuid: "82734084774e4dc18dcc948aecea3d17"
format_stamp: "Formatted at 2026-08-09 02:05:13 on dist-test-slave-ss4n"
server_key: "a3e0354dd9e3c45d933b0eab315b4c1e"
server_key_iv: "965f5f445a01f2f7611ff6cb9731a7fb"
server_key_version: "encryptionkey@0"
I20260809 02:05:13.940788 16454 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 02:05:13.943408 16471 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:05:13.944739 16454 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260809 02:05:13.944881 16454 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786240941968412-1795-0/minicluster-data/master-1/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786240941968412-1795-0/minicluster-data/master-1/wal
uuid: "82734084774e4dc18dcc948aecea3d17"
format_stamp: "Formatted at 2026-08-09 02:05:13 on dist-test-slave-ss4n"
server_key: "a3e0354dd9e3c45d933b0eab315b4c1e"
server_key_iv: "965f5f445a01f2f7611ff6cb9731a7fb"
server_key_version: "encryptionkey@0"
I20260809 02:05:13.945001 16454 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786240941968412-1795-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786240941968412-1795-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786240941968412-1795-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:05:13.956058 16454 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:05:13.959836 16454 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:05:13.960090 16454 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:05:13.968672 16454 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.253:39993
I20260809 02:05:13.968704 16524 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.253:39993 every 8 connection(s)
I20260809 02:05:13.969786 16454 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786240941968412-1795-0/minicluster-data/master-1/data/info.pb
I20260809 02:05:13.972491 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 16454
I20260809 02:05:13.972631 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786240941968412-1795-0/minicluster-data/master-1/wal/instance
I20260809 02:05:13.972635 16525 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:05:13.972949 1795 external_mini_cluster.cc:1428] Setting key 89ca1f67f3c9ee77b91124811b716634
I20260809 02:05:13.977159 16525 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.192.254" port: 43569 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.1.192.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.192.252@KRBTEST.COM" created.
Aug 09 02:05:13 dist-test-slave-ss4n krb5kdc[16361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241113, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.253@KRBTEST.COM for kudu/127.1.192.254@KRBTEST.COM
I20260809 02:05:13.991660 16449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:13.979158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36056 (local address 127.1.192.254:43569)
0809 02:05:13.979287 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0809 02:05:13.979291 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:05:13.980420 (+ 1129us) server_negotiation.cc:407] Connection header received
0809 02:05:13.981350 (+ 930us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:05:13.981354 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:05:13.981413 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:05:13.981669 (+ 256us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:05:13.982830 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:13.983420 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:13.984652 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:13.984963 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:13.985076 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:05:13.987595 (+ 2519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:05:13.987599 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:05:13.987601 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:05:13.987635 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:05:13.989182 (+ 1547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:05:13.989855 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:05:13.989859 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:05:13.989861 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:05:13.989951 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:05:13.990320 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:05:13.990325 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:05:13.990327 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:05:13.990608 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:05:13.990729 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0809 02:05:13.991296 (+ 567us) server_negotiation.cc:299] Negotiation successful
0809 02:05:13.991464 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 02:05:13.992542 16528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:13.979686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:43569 (local address 127.0.0.1:36056)
0809 02:05:13.980246 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:13.980292 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 02:05:13.981150 (+ 858us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:05:13.981830 (+ 680us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:05:13.981836 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:05:13.982220 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:05:13.982686 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:05:13.982700 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:05:13.983552 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:05:13.983556 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:05:13.984525 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:05:13.984535 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:05:13.985123 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:05:13.985127 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:05:13.985291 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:05:13.985955 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:05:13.985971 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:05:13.987404 (+ 1433us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:05:13.989334 (+ 1930us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:05:13.989341 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:05:13.989358 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:05:13.989733 (+ 375us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:05:13.990053 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:05:13.990057 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:05:13.990059 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:05:13.990216 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:05:13.990728 (+ 512us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:05:13.990735 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:05:13.991028 (+ 293us) client_negotiation.cc:769] Sending connection context
0809 02:05:13.991236 (+ 208us) client_negotiation.cc:240] Negotiation successful
0809 02:05:13.991605 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":167,"threads_started":1}
I20260809 02:05:13.994182 16525 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.192.253" port: 39993 } has no permanent_uuid. Determining permanent_uuid...
I20260809 02:05:13.994403 16446 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.192.253" port: 39993 } attempt: 3
Aug 09 02:05:14 dist-test-slave-ss4n krb5kdc[16361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241113, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.253@KRBTEST.COM for kudu/127.1.192.253@KRBTEST.COM
Aug 09 02:05:14 dist-test-slave-ss4n krb5kdc[16361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241113, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.254@KRBTEST.COM for kudu/127.1.192.253@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.192.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.252.keytab.
Entry for principal kudu/127.1.192.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.252.keytab.
I20260809 02:05:14.005716 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786240941968412-1795-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786240941968412-1795-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786240941968412-1795-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786240941968412-1795-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.192.252:46255
--webserver_interface=127.1.192.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.252.keytab
--principal=kudu/127.1.192.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:33853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.192.254:43569,127.1.192.253:39993,127.1.192.252:46255
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 02:05:14.006979 16528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:13.994908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.253:39993 (local address 127.0.0.1:33674)
0809 02:05:13.995158 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:13.995175 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 02:05:13.995307 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:05:13.996094 (+ 787us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:05:13.996097 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:05:13.996307 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:05:13.996521 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:05:13.996530 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:05:13.997704 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:05:13.997707 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:05:13.998542 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:05:13.998552 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:05:13.999360 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:05:13.999364 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:05:13.999488 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:05:14.000299 (+ 811us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:05:14.000319 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:05:14.002039 (+ 1720us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:05:14.005202 (+ 3163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:05:14.005209 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:05:14.005211 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:05:14.005417 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:05:14.005768 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:05:14.005771 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:05:14.005773 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:05:14.005894 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:05:14.006376 (+ 482us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:05:14.006380 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:05:14.006470 (+ 90us) client_negotiation.cc:769] Sending connection context
0809 02:05:14.006605 (+ 135us) client_negotiation.cc:240] Negotiation successful
0809 02:05:14.006762 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"mutex_wait_us":41}
I20260809 02:05:14.007074 16451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:13.994909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.253:39993 (local address 127.0.0.1:33672)
0809 02:05:13.995087 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:13.995143 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0809 02:05:13.995235 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:05:13.996284 (+ 1049us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:05:13.996288 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:05:13.996477 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:05:13.997030 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:05:13.997040 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:05:13.997992 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:05:13.997996 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:05:13.998814 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:05:13.998824 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:05:13.999360 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:05:13.999364 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:05:13.999555 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:05:14.000313 (+ 758us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:05:14.000329 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:05:14.002551 (+ 2222us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:05:14.004821 (+ 2270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:05:14.004825 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:05:14.004827 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:05:14.005113 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:05:14.005602 (+ 489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:05:14.005605 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:05:14.005608 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:05:14.005708 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:05:14.006391 (+ 683us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:05:14.006397 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:05:14.006522 (+ 125us) client_negotiation.cc:769] Sending connection context
0809 02:05:14.006629 (+ 107us) client_negotiation.cc:240] Negotiation successful
0809 02:05:14.006813 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103}
I20260809 02:05:14.007048 16530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:13.995019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33674 (local address 127.1.192.253:39993)
0809 02:05:13.995748 (+ 729us) server_negotiation.cc:206] Beginning negotiation
0809 02:05:13.995756 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 02:05:13.995775 (+ 19us) server_negotiation.cc:407] Connection header received
0809 02:05:13.995811 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:05:13.995816 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:05:13.995969 (+ 153us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:05:13.996479 (+ 510us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:05:13.996647 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:13.997593 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:13.998658 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:13.999207 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:13.999419 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:05:14.002231 (+ 2812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:05:14.002242 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:05:14.002256 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:05:14.002279 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:05:14.005079 (+ 2800us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:05:14.005571 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:05:14.005577 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:05:14.005579 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:05:14.005663 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:05:14.006046 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:05:14.006049 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:05:14.006051 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:05:14.006252 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:05:14.006412 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0809 02:05:14.006715 (+ 303us) server_negotiation.cc:299] Negotiation successful
0809 02:05:14.006857 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":97,"server-negotiator.queue_time_us":651,"spinlock_wait_cycles":11392,"thread_start_us":131,"threads_started":1}
I20260809 02:05:14.007766 16531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:13.995407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33672 (local address 127.1.192.253:39993)
0809 02:05:13.995781 (+ 374us) server_negotiation.cc:206] Beginning negotiation
0809 02:05:13.995834 (+ 53us) server_negotiation.cc:399] Waiting for connection header
0809 02:05:13.995850 (+ 16us) server_negotiation.cc:407] Connection header received
0809 02:05:13.996112 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:05:13.996120 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:05:13.996161 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:05:13.996291 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:05:13.997144 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:13.997800 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:13.998911 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:13.999207 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:13.999416 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:05:14.002729 (+ 3313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:05:14.002733 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:05:14.002737 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:05:14.002768 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:05:14.004702 (+ 1934us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:05:14.005275 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:05:14.005283 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:05:14.005285 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:05:14.005384 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:05:14.005864 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:05:14.005869 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:05:14.005874 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:05:14.006252 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:05:14.007293 (+ 1041us) server_negotiation.cc:1035] Waiting for connection context
0809 02:05:14.007380 (+ 87us) server_negotiation.cc:299] Negotiation successful
0809 02:05:14.007523 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":154,"server-negotiator.queue_time_us":311,"spinlock_wait_cycles":7936,"thread_start_us":158,"threads_started":1}
I20260809 02:05:14.009449 16446 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.192.252" port: 46255 } has no permanent_uuid. Determining permanent_uuid...
I20260809 02:05:14.009688 16525 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.192.252" port: 46255 } has no permanent_uuid. Determining permanent_uuid...
I20260809 02:05:14.010356 16451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:14.010070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.252:46255 (local address 127.0.0.1:36936)
0809 02:05:14.010247 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:14.010293 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.252:46255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
I20260809 02:05:14.010428 16528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:14.010079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.252:46255 (local address 127.0.0.1:36952)
0809 02:05:14.010239 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:14.010340 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.252:46255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260809 02:05:14.010701 16474 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.192.252:46255: connect: Connection refused (error 111)
I20260809 02:05:14.011286 16451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:14.010820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.252:46255 (local address 127.0.0.1:36968)
0809 02:05:14.011163 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:14.011212 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.252:46255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":110,"threads_started":1}
W20260809 02:05:14.011441 16446 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.192.252:46255: Network error: Client connection negotiation failed: client connection to 127.1.192.252:46255: connect: Connection refused (error 111)
I20260809 02:05:14.012082 16528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:14.011774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.252:46255 (local address 127.0.0.1:36982)
0809 02:05:14.011918 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:14.011992 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.252:46255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260809 02:05:14.012246 16525 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.192.252:46255: Network error: Client connection negotiation failed: client connection to 127.1.192.252:46255: connect: Connection refused (error 111)
I20260809 02:05:14.042616 16446 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.192.252" port: 46255 } attempt: 1
I20260809 02:05:14.043612 16451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:14.043290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.252:46255 (local address 127.0.0.1:36998)
0809 02:05:14.043457 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:14.043524 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.252:46255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260809 02:05:14.044513 16451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:14.044206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.252:46255 (local address 127.0.0.1:37014)
0809 02:05:14.044369 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:14.044421 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.252:46255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
W20260809 02:05:14.044762 16446 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.192.252:46255: Network error: Client connection negotiation failed: client connection to 127.1.192.252:46255: connect: Connection refused (error 111)
I20260809 02:05:14.061621 16525 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.192.252" port: 46255 } attempt: 1
I20260809 02:05:14.062598 16528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:14.062236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.252:46255 (local address 127.0.0.1:37030)
0809 02:05:14.062413 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:14.062490 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.252:46255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260809 02:05:14.063383 16528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:14.063147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.252:46255 (local address 127.0.0.1:37044)
0809 02:05:14.063255 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:14.063310 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.252:46255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260809 02:05:14.063603 16525 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.192.252:46255: Network error: Client connection negotiation failed: client connection to 127.1.192.252:46255: connect: Connection refused (error 111)
I20260809 02:05:14.119927 16446 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.192.252" port: 46255 } attempt: 2
I20260809 02:05:14.120893 16451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:14.120515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.252:46255 (local address 127.0.0.1:37060)
0809 02:05:14.120730 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:14.120803 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.252:46255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
I20260809 02:05:14.121730 16451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:14.121401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.252:46255 (local address 127.0.0.1:37062)
0809 02:05:14.121599 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:14.121658 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.252:46255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122}
W20260809 02:05:14.121891 16446 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.192.252:46255: Network error: Client connection negotiation failed: client connection to 127.1.192.252:46255: connect: Connection refused (error 111)
W20260809 02:05:14.126332 16532 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:05:14.126605 16532 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:05:14.126678 16532 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:05:14.130801 16532 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:05:14.130858 16532 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:05:14.130878 16532 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:05:14.130931 16532 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:05:14.130964 16532 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:05:14.131783 16525 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.192.252" port: 46255 } attempt: 2
I20260809 02:05:14.132589 16528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:14.132270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.252:46255 (local address 127.0.0.1:37066)
0809 02:05:14.132428 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:14.132495 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.252:46255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260809 02:05:14.133278 16528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:14.133066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.252:46255 (local address 127.0.0.1:37078)
0809 02:05:14.133168 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:14.133218 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.252:46255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260809 02:05:14.133412 16525 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.192.252:46255: Network error: Client connection negotiation failed: client connection to 127.1.192.252:46255: connect: Connection refused (error 111)
I20260809 02:05:14.136906 16532 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:33853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786240941968412-1795-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786240941968412-1795-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.1.192.254:43569,127.1.192.253:39993,127.1.192.252:46255
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.252.keytab
--principal=kudu/127.1.192.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.192.252:46255
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786240941968412-1795-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.1.192.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16532
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786240941968412-1795-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:05:14.138104 16532 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:05:14.139132 16532 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:05:14.147893 16543 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:05:14.147915 16539 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:05:14.147919 16540 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:05:14.148284 16532 server_base.cc:1034] running on GCE node
I20260809 02:05:14.148918 16532 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:05:14.150080 16532 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:05:14.151278 16532 hybrid_clock.cc:648] HybridClock initialized: now 1786241114151245 us; error 45 us; skew 500 ppm
Aug 09 02:05:14 dist-test-slave-ss4n krb5kdc[16361](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241114, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:05:14.154299 16532 init.cc:377] Logged in from keytab as kudu/127.1.192.252@KRBTEST.COM (short username kudu)
I20260809 02:05:14.155220 16532 webserver.cc:460] Webserver started at http://127.1.192.252:37571/ using document root <none> and password file <none>
I20260809 02:05:14.155735 16532 fs_manager.cc:359] Metadata directory not provided
I20260809 02:05:14.155795 16532 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:05:14.155956 16532 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:05:14.158064 16532 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786240941968412-1795-0/minicluster-data/master-2/data/instance:
uuid: "7bf98feef4cb44fd95aac44fd386337d"
format_stamp: "Formatted at 2026-08-09 02:05:14 on dist-test-slave-ss4n"
server_key: "699baa908cce5bc04dc39b4d24c1c71f"
server_key_iv: "36af343216bd3d6bd03b7df9b41a5877"
server_key_version: "encryptionkey@0"
I20260809 02:05:14.158624 16532 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786240941968412-1795-0/minicluster-data/master-2/wal/instance:
uuid: "7bf98feef4cb44fd95aac44fd386337d"
format_stamp: "Formatted at 2026-08-09 02:05:14 on dist-test-slave-ss4n"
server_key: "699baa908cce5bc04dc39b4d24c1c71f"
server_key_iv: "36af343216bd3d6bd03b7df9b41a5877"
server_key_version: "encryptionkey@0"
I20260809 02:05:14.162362 16532 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 02:05:14.165460 16551 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:05:14.166939 16532 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 02:05:14.167112 16532 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786240941968412-1795-0/minicluster-data/master-2/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786240941968412-1795-0/minicluster-data/master-2/wal
uuid: "7bf98feef4cb44fd95aac44fd386337d"
format_stamp: "Formatted at 2026-08-09 02:05:14 on dist-test-slave-ss4n"
server_key: "699baa908cce5bc04dc39b4d24c1c71f"
server_key_iv: "36af343216bd3d6bd03b7df9b41a5877"
server_key_version: "encryptionkey@0"
I20260809 02:05:14.167248 16532 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786240941968412-1795-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786240941968412-1795-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786240941968412-1795-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:05:14.179610 16532 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:05:14.183297 16532 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:05:14.183539 16532 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:05:14.191556 16532 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.252:46255
I20260809 02:05:14.191601 16605 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.252:46255 every 8 connection(s)
I20260809 02:05:14.192705 16532 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786240941968412-1795-0/minicluster-data/master-2/data/info.pb
I20260809 02:05:14.195482 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 16532
I20260809 02:05:14.195647 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786240941968412-1795-0/minicluster-data/master-2/wal/instance
I20260809 02:05:14.195837 16606 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:05:14.195972 1795 external_mini_cluster.cc:1428] Setting key 43b180baa6e471ea67e9b1670eebed35
I20260809 02:05:14.200974 16606 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.192.254" port: 43569 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 02:05:14 dist-test-slave-ss4n krb5kdc[16361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241113, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.192.252@KRBTEST.COM
Aug 09 02:05:14 dist-test-slave-ss4n krb5kdc[16361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241114, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.252@KRBTEST.COM for kudu/127.1.192.254@KRBTEST.COM
I20260809 02:05:14.211614 16610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:14.197358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37090 (local address 127.1.192.252:46255)
0809 02:05:14.197856 (+ 498us) server_negotiation.cc:206] Beginning negotiation
0809 02:05:14.197872 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 02:05:14.197910 (+ 38us) server_negotiation.cc:407] Connection header received
0809 02:05:14.198590 (+ 680us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:05:14.198606 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:05:14.198932 (+ 326us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:05:14.199211 (+ 279us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:05:14.200255 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:14.201533 (+ 1278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:14.202701 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:14.203426 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:14.204021 (+ 595us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:05:14.206272 (+ 2251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:05:14.206277 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:05:14.206286 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:05:14.206314 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:05:14.208344 (+ 2030us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:05:14.208829 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:05:14.208835 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:05:14.208841 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:05:14.208958 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:05:14.209248 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:05:14.209253 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:05:14.209255 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:05:14.209735 (+ 480us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:05:14.209956 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0809 02:05:14.210213 (+ 257us) server_negotiation.cc:299] Negotiation successful
0809 02:05:14.210541 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":327,"thread_start_us":199,"threads_started":1}
I20260809 02:05:14.212231 16449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:14.203078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36062 (local address 127.1.192.254:43569)
0809 02:05:14.203204 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0809 02:05:14.203208 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:05:14.203620 (+ 412us) server_negotiation.cc:407] Connection header received
0809 02:05:14.203734 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:05:14.203737 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:05:14.203792 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:05:14.203879 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:05:14.204691 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:14.205176 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:14.206164 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:14.206393 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:14.206496 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:05:14.208747 (+ 2251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:05:14.208750 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:05:14.208753 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:05:14.208786 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:05:14.210370 (+ 1584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:05:14.210904 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:05:14.210909 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:05:14.210911 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:05:14.211002 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:05:14.211327 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:05:14.211331 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:05:14.211332 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:05:14.211580 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:05:14.211664 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 02:05:14.212018 (+ 354us) server_negotiation.cc:299] Negotiation successful
0809 02:05:14.212120 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 02:05:14.212292 16611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:14.203022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:43569 (local address 127.0.0.1:36062)
0809 02:05:14.203438 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:14.203478 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 02:05:14.203626 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:05:14.203932 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:05:14.203939 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:05:14.204434 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:05:14.204567 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:05:14.204573 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:05:14.205294 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:05:14.205297 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:05:14.206041 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:05:14.206050 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:05:14.206512 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:05:14.206516 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:05:14.206638 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:05:14.207076 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:05:14.207087 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:05:14.208598 (+ 1511us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:05:14.210513 (+ 1915us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:05:14.210522 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:05:14.210527 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:05:14.210777 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:05:14.211107 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:05:14.211110 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:05:14.211112 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:05:14.211203 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:05:14.211673 (+ 470us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:05:14.211681 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:05:14.211801 (+ 120us) client_negotiation.cc:769] Sending connection context
0809 02:05:14.211961 (+ 160us) client_negotiation.cc:240] Negotiation successful
0809 02:05:14.212135 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":190,"threads_started":1}
I20260809 02:05:14.213761 16606 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.192.253" port: 39993 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 02:05:14 dist-test-slave-ss4n krb5kdc[16361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241114, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.252@KRBTEST.COM for kudu/127.1.192.253@KRBTEST.COM
Aug 09 02:05:14 dist-test-slave-ss4n krb5kdc[16361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241113, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.192.254@KRBTEST.COM
I20260809 02:05:14.221046 16446 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.192.252" port: 46255 } attempt: 3
I20260809 02:05:14.223515 16449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:14.214293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36070 (local address 127.1.192.254:43569)
0809 02:05:14.214413 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0809 02:05:14.214431 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0809 02:05:14.214449 (+ 18us) server_negotiation.cc:407] Connection header received
0809 02:05:14.214483 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:05:14.214486 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:05:14.214527 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:05:14.214911 (+ 384us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:05:14.215628 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:14.216098 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:14.216844 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:14.217056 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:14.217159 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:05:14.219809 (+ 2650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:05:14.219813 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:05:14.219816 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:05:14.219841 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:05:14.221439 (+ 1598us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:05:14.222059 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:05:14.222065 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:05:14.222066 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:05:14.222164 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:05:14.222507 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:05:14.222512 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:05:14.222514 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:05:14.222735 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:05:14.223001 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0809 02:05:14.223169 (+ 168us) server_negotiation.cc:299] Negotiation successful
0809 02:05:14.223294 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 02:05:14.223558 16531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:14.214539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33686 (local address 127.1.192.253:39993)
0809 02:05:14.214663 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0809 02:05:14.214672 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 02:05:14.214684 (+ 12us) server_negotiation.cc:407] Connection header received
0809 02:05:14.214719 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:05:14.214721 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:05:14.214754 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:05:14.214814 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:05:14.215300 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:14.215894 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:14.216716 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:14.216962 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:14.217079 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:05:14.219248 (+ 2169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:05:14.219251 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:05:14.219253 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:05:14.219279 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:05:14.221185 (+ 1906us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:05:14.222037 (+ 852us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:05:14.222041 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:05:14.222042 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:05:14.222146 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:05:14.222508 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:05:14.222511 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:05:14.222513 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:05:14.222768 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:05:14.222842 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 02:05:14.223246 (+ 404us) server_negotiation.cc:299] Negotiation successful
0809 02:05:14.223375 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 02:05:14.223986 16611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:14.214289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.253:39993 (local address 127.0.0.1:33686)
0809 02:05:14.214446 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:14.214458 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:05:14.214530 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:05:14.214839 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:05:14.214841 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:05:14.215016 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:05:14.215148 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:05:14.215155 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:05:14.216030 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:05:14.216033 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:05:14.216550 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:05:14.216557 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:05:14.217071 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:05:14.217075 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:05:14.217167 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:05:14.217672 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:05:14.217684 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:05:14.219046 (+ 1362us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:05:14.221325 (+ 2279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:05:14.221329 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:05:14.221331 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:05:14.221614 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:05:14.222279 (+ 665us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:05:14.222282 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:05:14.222284 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:05:14.222378 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:05:14.222925 (+ 547us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:05:14.222927 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:05:14.223035 (+ 108us) client_negotiation.cc:769] Sending connection context
0809 02:05:14.223697 (+ 662us) client_negotiation.cc:240] Negotiation successful
0809 02:05:14.223831 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91}
I20260809 02:05:14.224613 16525 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.192.252" port: 46255 } attempt: 3
I20260809 02:05:14.224859 16606 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.192.252" port: 46255 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 02:05:14 dist-test-slave-ss4n krb5kdc[16361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241113, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.254@KRBTEST.COM for kudu/127.1.192.252@KRBTEST.COM
Aug 09 02:05:14 dist-test-slave-ss4n krb5kdc[16361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241113, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.253@KRBTEST.COM for kudu/127.1.192.252@KRBTEST.COM
I20260809 02:05:14.231599 16451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:14.221654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.252:46255 (local address 127.0.0.1:37104)
0809 02:05:14.221792 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:14.221807 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 02:05:14.221883 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:05:14.222154 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:05:14.222157 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:05:14.222376 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:05:14.222507 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:05:14.222514 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:05:14.224061 (+ 1547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:05:14.224067 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:05:14.224807 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:05:14.224817 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:05:14.225376 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:05:14.225380 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:05:14.225461 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:05:14.226025 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:05:14.226042 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:05:14.227527 (+ 1485us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:05:14.229706 (+ 2179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:05:14.229710 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:05:14.229712 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:05:14.230014 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:05:14.230370 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:05:14.230373 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:05:14.230375 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:05:14.230471 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:05:14.231023 (+ 552us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:05:14.231026 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:05:14.231146 (+ 120us) client_negotiation.cc:769] Sending connection context
0809 02:05:14.231243 (+ 97us) client_negotiation.cc:240] Negotiation successful
0809 02:05:14.231395 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
Aug 09 02:05:14 dist-test-slave-ss4n krb5kdc[16361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241114, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.252@KRBTEST.COM for kudu/127.1.192.252@KRBTEST.COM
I20260809 02:05:14.232717 16610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:14.221710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37104 (local address 127.1.192.252:46255)
0809 02:05:14.221821 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0809 02:05:14.221827 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:05:14.221841 (+ 14us) server_negotiation.cc:407] Connection header received
0809 02:05:14.221974 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:05:14.221979 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:05:14.222029 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:05:14.222614 (+ 585us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:05:14.222703 (+ 89us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:14.223921 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:14.224946 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:14.225257 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:14.225385 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:05:14.227746 (+ 2361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:05:14.227750 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:05:14.227753 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:05:14.227778 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:05:14.229579 (+ 1801us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:05:14.230151 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:05:14.230156 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:05:14.230158 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:05:14.230241 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:05:14.230628 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:05:14.230631 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:05:14.230633 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:05:14.230860 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:05:14.232189 (+ 1329us) server_negotiation.cc:1035] Waiting for connection context
0809 02:05:14.232344 (+ 155us) server_negotiation.cc:299] Negotiation successful
0809 02:05:14.232522 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
Aug 09 02:05:14 dist-test-slave-ss4n krb5kdc[16361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241113, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.192.253@KRBTEST.COM
I20260809 02:05:14.236253 16528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:14.225014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.252:46255 (local address 127.0.0.1:37106)
0809 02:05:14.225147 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:14.225161 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 02:05:14.225234 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:05:14.226461 (+ 1227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:05:14.226464 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:05:14.226672 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:05:14.226772 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:05:14.226779 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:05:14.228000 (+ 1221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:05:14.228006 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:05:14.228863 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:05:14.228875 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:05:14.229688 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:05:14.229692 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:05:14.229794 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:05:14.230296 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:05:14.230317 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:05:14.231988 (+ 1671us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:05:14.234519 (+ 2531us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:05:14.234524 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:05:14.234526 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:05:14.234797 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:05:14.235103 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:05:14.235107 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:05:14.235109 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:05:14.235203 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:05:14.235694 (+ 491us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:05:14.235698 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:05:14.235798 (+ 100us) client_negotiation.cc:769] Sending connection context
0809 02:05:14.235907 (+ 109us) client_negotiation.cc:240] Negotiation successful
0809 02:05:14.236075 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260809 02:05:14.236277 16613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:14.225244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37106 (local address 127.1.192.252:46255)
0809 02:05:14.226212 (+ 968us) server_negotiation.cc:206] Beginning negotiation
0809 02:05:14.226222 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 02:05:14.226238 (+ 16us) server_negotiation.cc:407] Connection header received
0809 02:05:14.226271 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:05:14.226273 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:05:14.226311 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:05:14.226908 (+ 597us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:05:14.227059 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:14.227851 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:14.229037 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:14.229525 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:14.230030 (+ 505us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:05:14.232153 (+ 2123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:05:14.232159 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:05:14.232163 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:05:14.232191 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:05:14.234389 (+ 2198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:05:14.234893 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:05:14.234898 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:05:14.234900 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:05:14.234992 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:05:14.235302 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:05:14.235307 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:05:14.235309 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:05:14.235589 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:05:14.235676 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0809 02:05:14.235978 (+ 302us) server_negotiation.cc:299] Negotiation successful
0809 02:05:14.236098 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":893,"spinlock_wait_cycles":1280,"thread_start_us":148,"threads_started":1}
I20260809 02:05:14.236595 16612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:14.225620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37108 (local address 127.1.192.252:46255)
0809 02:05:14.226820 (+ 1200us) server_negotiation.cc:206] Beginning negotiation
0809 02:05:14.226832 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 02:05:14.226847 (+ 15us) server_negotiation.cc:407] Connection header received
0809 02:05:14.226884 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:05:14.226888 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:05:14.226935 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:05:14.227416 (+ 481us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:05:14.227514 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:14.228278 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:14.229333 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:14.229723 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:14.230085 (+ 362us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:05:14.232695 (+ 2610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:05:14.232698 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:05:14.232701 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:05:14.232723 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:05:14.234670 (+ 1947us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:05:14.235142 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:05:14.235145 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:05:14.235147 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:05:14.235223 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:05:14.235570 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:05:14.235573 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:05:14.235575 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:05:14.235796 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:05:14.235891 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 02:05:14.236355 (+ 464us) server_negotiation.cc:299] Negotiation successful
0809 02:05:14.236449 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1118,"thread_start_us":117,"threads_started":1}
I20260809 02:05:14.236789 16611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:14.225288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.252:46255 (local address 127.0.0.1:37108)
0809 02:05:14.225750 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:14.225764 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 02:05:14.225862 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:05:14.227056 (+ 1194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:05:14.227059 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:05:14.227232 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:05:14.227321 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:05:14.227328 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:05:14.228401 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:05:14.228404 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:05:14.229208 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:05:14.229219 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:05:14.229826 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:05:14.229829 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:05:14.229899 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:05:14.230445 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:05:14.230460 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:05:14.232552 (+ 2092us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:05:14.234808 (+ 2256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:05:14.234811 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:05:14.234813 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:05:14.235040 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:05:14.235315 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:05:14.235321 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:05:14.235326 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:05:14.235465 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:05:14.235926 (+ 461us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:05:14.235929 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:05:14.236001 (+ 72us) client_negotiation.cc:769] Sending connection context
0809 02:05:14.236098 (+ 97us) client_negotiation.cc:240] Negotiation successful
0809 02:05:14.236221 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":385}
I20260809 02:05:14.236821 16446 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9815451a73d548b8acc0309c51c4a3ca: Bootstrap starting.
I20260809 02:05:14.239966 16446 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9815451a73d548b8acc0309c51c4a3ca: Neither blocks nor log segments found. Creating new log.
I20260809 02:05:14.240314 16606 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7bf98feef4cb44fd95aac44fd386337d: Bootstrap starting.
I20260809 02:05:14.240878 16446 log.cc:824] T 00000000000000000000000000000000 P 9815451a73d548b8acc0309c51c4a3ca: Log is configured to *not* fsync() on all Append() calls
I20260809 02:05:14.240999 16525 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82734084774e4dc18dcc948aecea3d17: Bootstrap starting.
I20260809 02:05:14.242112 16531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:14.225842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33702 (local address 127.1.192.253:39993)
0809 02:05:14.225948 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0809 02:05:14.225955 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 02:05:14.226213 (+ 258us) server_negotiation.cc:407] Connection header received
0809 02:05:14.226259 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:05:14.226262 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:05:14.226306 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:05:14.226397 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:05:14.227235 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:14.227990 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:14.229666 (+ 1676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:14.230446 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:14.230532 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:05:14.238123 (+ 7591us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:05:14.238127 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:05:14.238130 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:05:14.238155 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:05:14.239879 (+ 1724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:05:14.240492 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:05:14.240495 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:05:14.240497 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:05:14.240599 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:05:14.240926 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:05:14.240930 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:05:14.240931 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:05:14.241208 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:05:14.241364 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0809 02:05:14.241807 (+ 443us) server_negotiation.cc:299] Negotiation successful
0809 02:05:14.241926 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 02:05:14.243036 16606 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7bf98feef4cb44fd95aac44fd386337d: Neither blocks nor log segments found. Creating new log.
I20260809 02:05:14.243489 16446 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9815451a73d548b8acc0309c51c4a3ca: No bootstrap required, opened a new log
I20260809 02:05:14.243927 16606 log.cc:824] T 00000000000000000000000000000000 P 7bf98feef4cb44fd95aac44fd386337d: Log is configured to *not* fsync() on all Append() calls
I20260809 02:05:14.244192 16525 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 82734084774e4dc18dcc948aecea3d17: Neither blocks nor log segments found. Creating new log.
I20260809 02:05:14.245219 16525 log.cc:824] T 00000000000000000000000000000000 P 82734084774e4dc18dcc948aecea3d17: Log is configured to *not* fsync() on all Append() calls
I20260809 02:05:14.246153 1795 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 02:05:14.247493 16525 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82734084774e4dc18dcc948aecea3d17: No bootstrap required, opened a new log
I20260809 02:05:14.247506 16446 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9815451a73d548b8acc0309c51c4a3ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9815451a73d548b8acc0309c51c4a3ca" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 43569 } } peers { permanent_uuid: "82734084774e4dc18dcc948aecea3d17" member_type: VOTER last_known_addr { host: "127.1.192.253" port: 39993 } } peers { permanent_uuid: "7bf98feef4cb44fd95aac44fd386337d" member_type: VOTER last_known_addr { host: "127.1.192.252" port: 46255 } }
I20260809 02:05:14.247807 16446 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9815451a73d548b8acc0309c51c4a3ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:05:14.247864 16446 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9815451a73d548b8acc0309c51c4a3ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9815451a73d548b8acc0309c51c4a3ca, State: Initialized, Role: FOLLOWER
I20260809 02:05:14.248363 16446 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9815451a73d548b8acc0309c51c4a3ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9815451a73d548b8acc0309c51c4a3ca" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 43569 } } peers { permanent_uuid: "82734084774e4dc18dcc948aecea3d17" member_type: VOTER last_known_addr { host: "127.1.192.253" port: 39993 } } peers { permanent_uuid: "7bf98feef4cb44fd95aac44fd386337d" member_type: VOTER last_known_addr { host: "127.1.192.252" port: 46255 } }
I20260809 02:05:14.249943 16446 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9815451a73d548b8acc0309c51c4a3ca [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:05:14.259160 16612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:14.248956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37114 (local address 127.1.192.252:46255)
0809 02:05:14.249061 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0809 02:05:14.249066 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:05:14.249275 (+ 209us) server_negotiation.cc:407] Connection header received
0809 02:05:14.249542 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:05:14.249546 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:05:14.249603 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:05:14.249657 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:05:14.250648 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:14.251434 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:14.252640 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:14.252979 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:14.253060 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:05:14.255013 (+ 1953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:05:14.255017 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:05:14.255019 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:05:14.255059 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:05:14.256968 (+ 1909us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:05:14.257726 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:05:14.257730 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:05:14.257731 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:05:14.257820 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:05:14.258139 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:05:14.258142 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:05:14.258144 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:05:14.258359 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:05:14.258499 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0809 02:05:14.258864 (+ 365us) server_negotiation.cc:299] Negotiation successful
0809 02:05:14.259000 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 02:05:14.254966 16374 master_runner.cc:186] Error getting master registration for 127.1.192.253:39993: OK, instance_id { permanent_uuid: "82734084774e4dc18dcc948aecea3d17" instance_seqno: 1786241113966176 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 02:05:14.259503 16374 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 02:05:14.252044 16525 raft_consensus.cc:348] T 00000000000000000000000000000000 P 82734084774e4dc18dcc948aecea3d17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9815451a73d548b8acc0309c51c4a3ca" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 43569 } } peers { permanent_uuid: "82734084774e4dc18dcc948aecea3d17" member_type: VOTER last_known_addr { host: "127.1.192.253" port: 39993 } } peers { permanent_uuid: "7bf98feef4cb44fd95aac44fd386337d" member_type: VOTER last_known_addr { host: "127.1.192.252" port: 46255 } }
I20260809 02:05:14.259842 16525 raft_consensus.cc:374] T 00000000000000000000000000000000 P 82734084774e4dc18dcc948aecea3d17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:05:14.259909 16525 raft_consensus.cc:729] T 00000000000000000000000000000000 P 82734084774e4dc18dcc948aecea3d17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82734084774e4dc18dcc948aecea3d17, State: Initialized, Role: FOLLOWER
I20260809 02:05:14.260298 16449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:14.249183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36072 (local address 127.1.192.254:43569)
0809 02:05:14.249325 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0809 02:05:14.249329 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:05:14.249348 (+ 19us) server_negotiation.cc:407] Connection header received
0809 02:05:14.249393 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:05:14.249395 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:05:14.249441 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:05:14.249700 (+ 259us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:05:14.251205 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:14.251975 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:14.253120 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:14.253408 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:14.253528 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:05:14.255255 (+ 1727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:05:14.255258 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:05:14.255261 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:05:14.255543 (+ 282us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:05:14.257458 (+ 1915us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:05:14.258342 (+ 884us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:05:14.258346 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:05:14.258348 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:05:14.258439 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:05:14.258781 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:05:14.258785 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:05:14.258786 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:05:14.259002 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:05:14.259101 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 02:05:14.260012 (+ 911us) server_negotiation.cc:299] Negotiation successful
0809 02:05:14.260143 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 02:05:14.260447 16525 consensus_queue.cc:260] T 00000000000000000000000000000000 P 82734084774e4dc18dcc948aecea3d17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9815451a73d548b8acc0309c51c4a3ca" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 43569 } } peers { permanent_uuid: "82734084774e4dc18dcc948aecea3d17" member_type: VOTER last_known_addr { host: "127.1.192.253" port: 39993 } } peers { permanent_uuid: "7bf98feef4cb44fd95aac44fd386337d" member_type: VOTER last_known_addr { host: "127.1.192.252" port: 46255 } }
W20260809 02:05:14.255561 16636 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9815451a73d548b8acc0309c51c4a3ca: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 02:05:14.261617 16636 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 02:05:14.261963 16525 sys_catalog.cc:564] T 00000000000000000000000000000000 P 82734084774e4dc18dcc948aecea3d17 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:05:14.262411 16624 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9815451a73d548b8acc0309c51c4a3ca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9815451a73d548b8acc0309c51c4a3ca" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 43569 } } peers { permanent_uuid: "82734084774e4dc18dcc948aecea3d17" member_type: VOTER last_known_addr { host: "127.1.192.253" port: 39993 } } peers { permanent_uuid: "7bf98feef4cb44fd95aac44fd386337d" member_type: VOTER last_known_addr { host: "127.1.192.252" port: 46255 } } }
I20260809 02:05:14.262619 16624 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9815451a73d548b8acc0309c51c4a3ca [sys.catalog]: This master's current role is: FOLLOWER
I20260809 02:05:14.262768 16624 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9815451a73d548b8acc0309c51c4a3ca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:05:14.262918 16624 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9815451a73d548b8acc0309c51c4a3ca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9815451a73d548b8acc0309c51c4a3ca" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 43569 } } peers { permanent_uuid: "82734084774e4dc18dcc948aecea3d17" member_type: VOTER last_known_addr { host: "127.1.192.253" port: 39993 } } peers { permanent_uuid: "7bf98feef4cb44fd95aac44fd386337d" member_type: VOTER last_known_addr { host: "127.1.192.252" port: 46255 } }
I20260809 02:05:14.263700 16606 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7bf98feef4cb44fd95aac44fd386337d: No bootstrap required, opened a new log
I20260809 02:05:14.265547 16624 leader_election.cc:288] T 00000000000000000000000000000000 P 9815451a73d548b8acc0309c51c4a3ca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 82734084774e4dc18dcc948aecea3d17 (127.1.192.253:39993), 7bf98feef4cb44fd95aac44fd386337d (127.1.192.252:46255)
I20260809 02:05:14.264437 16498 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9815451a73d548b8acc0309c51c4a3ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "82734084774e4dc18dcc948aecea3d17" is_pre_election: true
I20260809 02:05:14.266085 16498 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 82734084774e4dc18dcc948aecea3d17 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9815451a73d548b8acc0309c51c4a3ca in term 0.
I20260809 02:05:14.266517 16393 leader_election.cc:302] T 00000000000000000000000000000000 P 9815451a73d548b8acc0309c51c4a3ca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 82734084774e4dc18dcc948aecea3d17, 9815451a73d548b8acc0309c51c4a3ca; no voters:
I20260809 02:05:14.267040 16624 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9815451a73d548b8acc0309c51c4a3ca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:05:14.267181 16624 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9815451a73d548b8acc0309c51c4a3ca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:05:14.267251 16624 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9815451a73d548b8acc0309c51c4a3ca [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:05:14.268520 16624 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9815451a73d548b8acc0309c51c4a3ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9815451a73d548b8acc0309c51c4a3ca" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 43569 } } peers { permanent_uuid: "82734084774e4dc18dcc948aecea3d17" member_type: VOTER last_known_addr { host: "127.1.192.253" port: 39993 } } peers { permanent_uuid: "7bf98feef4cb44fd95aac44fd386337d" member_type: VOTER last_known_addr { host: "127.1.192.252" port: 46255 } }
I20260809 02:05:14.268913 16606 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7bf98feef4cb44fd95aac44fd386337d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9815451a73d548b8acc0309c51c4a3ca" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 43569 } } peers { permanent_uuid: "82734084774e4dc18dcc948aecea3d17" member_type: VOTER last_known_addr { host: "127.1.192.253" port: 39993 } } peers { permanent_uuid: "7bf98feef4cb44fd95aac44fd386337d" member_type: VOTER last_known_addr { host: "127.1.192.252" port: 46255 } }
I20260809 02:05:14.269201 16606 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7bf98feef4cb44fd95aac44fd386337d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:05:14.269304 16606 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7bf98feef4cb44fd95aac44fd386337d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bf98feef4cb44fd95aac44fd386337d, State: Initialized, Role: FOLLOWER
I20260809 02:05:14.269901 16606 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7bf98feef4cb44fd95aac44fd386337d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9815451a73d548b8acc0309c51c4a3ca" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 43569 } } peers { permanent_uuid: "82734084774e4dc18dcc948aecea3d17" member_type: VOTER last_known_addr { host: "127.1.192.253" port: 39993 } } peers { permanent_uuid: "7bf98feef4cb44fd95aac44fd386337d" member_type: VOTER last_known_addr { host: "127.1.192.252" port: 46255 } }
I20260809 02:05:14.269997 16624 leader_election.cc:288] T 00000000000000000000000000000000 P 9815451a73d548b8acc0309c51c4a3ca [CANDIDATE]: Term 1 election: Requested vote from peers 82734084774e4dc18dcc948aecea3d17 (127.1.192.253:39993), 7bf98feef4cb44fd95aac44fd386337d (127.1.192.252:46255)
I20260809 02:05:14.270884 16531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:14.250727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33714 (local address 127.1.192.253:39993)
0809 02:05:14.250859 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0809 02:05:14.250864 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:05:14.260717 (+ 9853us) server_negotiation.cc:407] Connection header received
0809 02:05:14.260901 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:05:14.260905 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:05:14.260951 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:05:14.260998 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:05:14.261872 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:14.262620 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:14.263792 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:14.264157 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:14.264233 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:05:14.266076 (+ 1843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:05:14.266080 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:05:14.266083 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:05:14.266112 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:05:14.268010 (+ 1898us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:05:14.268778 (+ 768us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:05:14.268782 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:05:14.268787 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:05:14.268890 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:05:14.269539 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:05:14.269543 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:05:14.269545 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:05:14.269806 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:05:14.269926 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0809 02:05:14.270599 (+ 673us) server_negotiation.cc:299] Negotiation successful
0809 02:05:14.270737 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 02:05:14.271453 16606 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7bf98feef4cb44fd95aac44fd386337d [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:05:14.271481 16579 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9815451a73d548b8acc0309c51c4a3ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bf98feef4cb44fd95aac44fd386337d" is_pre_election: true
I20260809 02:05:14.271508 16498 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9815451a73d548b8acc0309c51c4a3ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "82734084774e4dc18dcc948aecea3d17"
I20260809 02:05:14.271637 16498 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 82734084774e4dc18dcc948aecea3d17 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:05:14.271752 16579 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7bf98feef4cb44fd95aac44fd386337d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9815451a73d548b8acc0309c51c4a3ca in term 0.
I20260809 02:05:14.272960 16498 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 82734084774e4dc18dcc948aecea3d17 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9815451a73d548b8acc0309c51c4a3ca in term 1.
I20260809 02:05:14.273171 16580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9815451a73d548b8acc0309c51c4a3ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bf98feef4cb44fd95aac44fd386337d"
I20260809 02:05:14.273280 16393 leader_election.cc:302] T 00000000000000000000000000000000 P 9815451a73d548b8acc0309c51c4a3ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 82734084774e4dc18dcc948aecea3d17, 9815451a73d548b8acc0309c51c4a3ca; no voters:
I20260809 02:05:14.273301 16580 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7bf98feef4cb44fd95aac44fd386337d [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:05:14.273501 16624 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9815451a73d548b8acc0309c51c4a3ca [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:05:14.273612 16624 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9815451a73d548b8acc0309c51c4a3ca [term 1 LEADER]: Becoming Leader. State: Replica: 9815451a73d548b8acc0309c51c4a3ca, State: Running, Role: LEADER
I20260809 02:05:14.273921 16624 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9815451a73d548b8acc0309c51c4a3ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9815451a73d548b8acc0309c51c4a3ca" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 43569 } } peers { permanent_uuid: "82734084774e4dc18dcc948aecea3d17" member_type: VOTER last_known_addr { host: "127.1.192.253" port: 39993 } } peers { permanent_uuid: "7bf98feef4cb44fd95aac44fd386337d" member_type: VOTER last_known_addr { host: "127.1.192.252" port: 46255 } }
I20260809 02:05:14.274622 16580 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7bf98feef4cb44fd95aac44fd386337d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9815451a73d548b8acc0309c51c4a3ca in term 1.
I20260809 02:05:14.278283 16624 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9815451a73d548b8acc0309c51c4a3ca [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9815451a73d548b8acc0309c51c4a3ca. Latest consensus state: current_term: 1 leader_uuid: "9815451a73d548b8acc0309c51c4a3ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9815451a73d548b8acc0309c51c4a3ca" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 43569 } } peers { permanent_uuid: "82734084774e4dc18dcc948aecea3d17" member_type: VOTER last_known_addr { host: "127.1.192.253" port: 39993 } } peers { permanent_uuid: "7bf98feef4cb44fd95aac44fd386337d" member_type: VOTER last_known_addr { host: "127.1.192.252" port: 46255 } } }
I20260809 02:05:14.278424 16624 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9815451a73d548b8acc0309c51c4a3ca [sys.catalog]: This master's current role is: LEADER
W20260809 02:05:14.273366 16652 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 82734084774e4dc18dcc948aecea3d17: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260809 02:05:14.278345 16454 master_runner.cc:263] No leader master found from master 82734084774e4dc18dcc948aecea3d17
W20260809 02:05:14.281474 16652 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 02:05:14.281525 16454 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 02:05:14.281586 16653 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7bf98feef4cb44fd95aac44fd386337d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9815451a73d548b8acc0309c51c4a3ca" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 43569 } } peers { permanent_uuid: "82734084774e4dc18dcc948aecea3d17" member_type: VOTER last_known_addr { host: "127.1.192.253" port: 39993 } } peers { permanent_uuid: "7bf98feef4cb44fd95aac44fd386337d" member_type: VOTER last_known_addr { host: "127.1.192.252" port: 46255 } } }
I20260809 02:05:14.281710 16653 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7bf98feef4cb44fd95aac44fd386337d [sys.catalog]: This master's current role is: FOLLOWER
I20260809 02:05:14.281708 16640 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82734084774e4dc18dcc948aecea3d17 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9815451a73d548b8acc0309c51c4a3ca" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 43569 } } peers { permanent_uuid: "82734084774e4dc18dcc948aecea3d17" member_type: VOTER last_known_addr { host: "127.1.192.253" port: 39993 } } peers { permanent_uuid: "7bf98feef4cb44fd95aac44fd386337d" member_type: VOTER last_known_addr { host: "127.1.192.252" port: 46255 } } }
I20260809 02:05:14.281810 16640 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82734084774e4dc18dcc948aecea3d17 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 02:05:14.285990 16667 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:05:14.287024 16667 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:05:14.293018 16580 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7bf98feef4cb44fd95aac44fd386337d [term 1 FOLLOWER]: Refusing update from remote peer 9815451a73d548b8acc0309c51c4a3ca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:05:14.293144 16498 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 82734084774e4dc18dcc948aecea3d17 [term 1 FOLLOWER]: Refusing update from remote peer 9815451a73d548b8acc0309c51c4a3ca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:05:14.294528 16624 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9815451a73d548b8acc0309c51c4a3ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "82734084774e4dc18dcc948aecea3d17" member_type: VOTER last_known_addr { host: "127.1.192.253" port: 39993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 02:05:14.295091 16665 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7bf98feef4cb44fd95aac44fd386337d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 02:05:14.295183 16665 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 02:05:14.299350 16640 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82734084774e4dc18dcc948aecea3d17 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9815451a73d548b8acc0309c51c4a3ca. Latest consensus state: current_term: 1 leader_uuid: "9815451a73d548b8acc0309c51c4a3ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9815451a73d548b8acc0309c51c4a3ca" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 43569 } } peers { permanent_uuid: "82734084774e4dc18dcc948aecea3d17" member_type: VOTER last_known_addr { host: "127.1.192.253" port: 39993 } } peers { permanent_uuid: "7bf98feef4cb44fd95aac44fd386337d" member_type: VOTER last_known_addr { host: "127.1.192.252" port: 46255 } } }
I20260809 02:05:14.299515 16640 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82734084774e4dc18dcc948aecea3d17 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 02:05:14.299681 16624 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9815451a73d548b8acc0309c51c4a3ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bf98feef4cb44fd95aac44fd386337d" member_type: VOTER last_known_addr { host: "127.1.192.252" port: 46255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:05:14.307179 16624 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9815451a73d548b8acc0309c51c4a3ca [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9815451a73d548b8acc0309c51c4a3ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9815451a73d548b8acc0309c51c4a3ca" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 43569 } } peers { permanent_uuid: "82734084774e4dc18dcc948aecea3d17" member_type: VOTER last_known_addr { host: "127.1.192.253" port: 39993 } } peers { permanent_uuid: "7bf98feef4cb44fd95aac44fd386337d" member_type: VOTER last_known_addr { host: "127.1.192.252" port: 46255 } } }
I20260809 02:05:14.307336 16624 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9815451a73d548b8acc0309c51c4a3ca [sys.catalog]: This master's current role is: LEADER
I20260809 02:05:14.307446 16668 mvcc.cc:205] Tried to move back new op lower bound from 7316443604135301120 to 7316443604069265408. Current Snapshot: MvccSnapshot[applied={T|T < 7316443604135301120}]
I20260809 02:05:14.308828 16670 mvcc.cc:205] Tried to move back new op lower bound from 7316443604135301120 to 7316443604069265408. Current Snapshot: MvccSnapshot[applied={T|T < 7316443604135301120}]
I20260809 02:05:14.309077 16640 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82734084774e4dc18dcc948aecea3d17 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9815451a73d548b8acc0309c51c4a3ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9815451a73d548b8acc0309c51c4a3ca" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 43569 } } peers { permanent_uuid: "82734084774e4dc18dcc948aecea3d17" member_type: VOTER last_known_addr { host: "127.1.192.253" port: 39993 } } peers { permanent_uuid: "7bf98feef4cb44fd95aac44fd386337d" member_type: VOTER last_known_addr { host: "127.1.192.252" port: 46255 } } }
I20260809 02:05:14.309194 16640 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82734084774e4dc18dcc948aecea3d17 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 02:05:14.309588 16653 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7bf98feef4cb44fd95aac44fd386337d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9815451a73d548b8acc0309c51c4a3ca. Latest consensus state: current_term: 1 leader_uuid: "9815451a73d548b8acc0309c51c4a3ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9815451a73d548b8acc0309c51c4a3ca" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 43569 } } peers { permanent_uuid: "82734084774e4dc18dcc948aecea3d17" member_type: VOTER last_known_addr { host: "127.1.192.253" port: 39993 } } peers { permanent_uuid: "7bf98feef4cb44fd95aac44fd386337d" member_type: VOTER last_known_addr { host: "127.1.192.252" port: 46255 } } }
I20260809 02:05:14.309742 16653 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7bf98feef4cb44fd95aac44fd386337d [sys.catalog]: This master's current role is: FOLLOWER
I20260809 02:05:14.315384 16638 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9815451a73d548b8acc0309c51c4a3ca [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9815451a73d548b8acc0309c51c4a3ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9815451a73d548b8acc0309c51c4a3ca" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 43569 } } peers { permanent_uuid: "82734084774e4dc18dcc948aecea3d17" member_type: VOTER last_known_addr { host: "127.1.192.253" port: 39993 } } peers { permanent_uuid: "7bf98feef4cb44fd95aac44fd386337d" member_type: VOTER last_known_addr { host: "127.1.192.252" port: 46255 } } }
I20260809 02:05:14.315543 16638 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9815451a73d548b8acc0309c51c4a3ca [sys.catalog]: This master's current role is: LEADER
I20260809 02:05:14.316519 16653 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7bf98feef4cb44fd95aac44fd386337d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9815451a73d548b8acc0309c51c4a3ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9815451a73d548b8acc0309c51c4a3ca" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 43569 } } peers { permanent_uuid: "82734084774e4dc18dcc948aecea3d17" member_type: VOTER last_known_addr { host: "127.1.192.253" port: 39993 } } peers { permanent_uuid: "7bf98feef4cb44fd95aac44fd386337d" member_type: VOTER last_known_addr { host: "127.1.192.252" port: 46255 } } }
I20260809 02:05:14.316670 16653 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7bf98feef4cb44fd95aac44fd386337d [sys.catalog]: This master's current role is: FOLLOWER
I20260809 02:05:14.317982 16667 catalog_manager.cc:1347] Generated new cluster ID: 5969520b53224b4a8645f42f1b4bcffb
I20260809 02:05:14.318058 16667 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:05:14.318629 16672 mvcc.cc:205] Tried to move back new op lower bound from 7316443604135301120 to 7316443604069265408. Current Snapshot: MvccSnapshot[applied={T|T < 7316443604135301120}]
I20260809 02:05:14.352633 16667 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:05:14.354048 16667 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:05:14.373392 16667 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9815451a73d548b8acc0309c51c4a3ca: Generated new TSK 0
I20260809 02:05:14.374487 16667 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:05:15.283020 16652 catalog_manager.cc:1259] Loaded cluster ID: 5969520b53224b4a8645f42f1b4bcffb
I20260809 02:05:15.283134 16652 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 82734084774e4dc18dcc948aecea3d17: loading cluster ID for follower catalog manager: success
I20260809 02:05:15.284842 16652 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 82734084774e4dc18dcc948aecea3d17: acquiring CA information for follower catalog manager: success
I20260809 02:05:15.286149 16652 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 82734084774e4dc18dcc948aecea3d17: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 02:05:15.296639 16665 catalog_manager.cc:1259] Loaded cluster ID: 5969520b53224b4a8645f42f1b4bcffb
I20260809 02:05:15.296737 16665 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 7bf98feef4cb44fd95aac44fd386337d: loading cluster ID for follower catalog manager: success
I20260809 02:05:15.298549 16665 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 7bf98feef4cb44fd95aac44fd386337d: acquiring CA information for follower catalog manager: success
I20260809 02:05:15.299726 16665 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 7bf98feef4cb44fd95aac44fd386337d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 02:05:15.448305 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 16374
I20260809 02:05:15.456858 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 16454
I20260809 02:05:15.464932 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 16532
2026-08-09T02:05:15Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3167 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786240941968412-1795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:05:15 dist-test-slave-ss4n krb5kdc[16685](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:05:15 dist-test-slave-ss4n krb5kdc[16685](info): set up 2 sockets
Aug 09 02:05:15 dist-test-slave-ss4n krb5kdc[16685](info): commencing operation
krb5kdc: starting...
W20260809 02:05:16.612897 1795 mini_kdc.cc:121] Time spent starting KDC: real 1.123s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786240941968412-1795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786240941968412-1795-0/krb5kdc/test-user.keytab.
Aug 09 02:05:16 dist-test-slave-ss4n krb5kdc[16685](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241116, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:05:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:05:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.192.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.192.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.192.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab.
Entry for principal kudu/127.1.192.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab.
I20260809 02:05:16.766422 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786240941968412-1795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786240941968412-1795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786240941968412-1795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.192.254:33695
--webserver_interface=127.1.192.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab
--principal=kudu/127.1.192.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:36649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.192.254:33695
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:05:16.886260 16698 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:05:16.886624 16698 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:05:16.886725 16698 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:05:16.890973 16698 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:05:16.891072 16698 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:05:16.891111 16698 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:05:16.891150 16698 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:05:16.891191 16698 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:05:16.896761 16698 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:36649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786240941968412-1795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786240941968412-1795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.192.254:33695
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab
--principal=kudu/127.1.192.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.192.254:33695
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.192.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16698
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786240941968412-1795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:05:16.897948 16698 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:05:16.899003 16698 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:05:16.908108 16706 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:05:16.908083 16703 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:05:16.908219 16698 server_base.cc:1034] running on GCE node
W20260809 02:05:16.908082 16704 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:05:16.909144 16698 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:05:16.910494 16698 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:05:16.911722 16698 hybrid_clock.cc:648] HybridClock initialized: now 1786241116911687 us; error 53 us; skew 500 ppm
Aug 09 02:05:16 dist-test-slave-ss4n krb5kdc[16685](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241116, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:05:16.915334 16698 init.cc:377] Logged in from keytab as kudu/127.1.192.254@KRBTEST.COM (short username kudu)
I20260809 02:05:16.916414 16698 webserver.cc:460] Webserver started at http://127.1.192.254:33023/ using document root <none> and password file <none>
I20260809 02:05:16.917063 16698 fs_manager.cc:359] Metadata directory not provided
I20260809 02:05:16.917132 16698 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:05:16.917392 16698 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:05:16.919783 16698 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786240941968412-1795-0/minicluster-data/master-0/data/instance:
uuid: "3bc8638db8e5452a8dd40ac441c2e4c2"
format_stamp: "Formatted at 2026-08-09 02:05:16 on dist-test-slave-ss4n"
server_key: "75b2d6dd6abf5abc3c2fa52e41238e5f"
server_key_iv: "6cf0d8437d9d796d6afe28f097ec5a2a"
server_key_version: "encryptionkey@0"
I20260809 02:05:16.920445 16698 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786240941968412-1795-0/minicluster-data/master-0/wal/instance:
uuid: "3bc8638db8e5452a8dd40ac441c2e4c2"
format_stamp: "Formatted at 2026-08-09 02:05:16 on dist-test-slave-ss4n"
server_key: "75b2d6dd6abf5abc3c2fa52e41238e5f"
server_key_iv: "6cf0d8437d9d796d6afe28f097ec5a2a"
server_key_version: "encryptionkey@0"
I20260809 02:05:16.924844 16698 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 02:05:16.927753 16713 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:05:16.929160 16698 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 02:05:16.929430 16698 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786240941968412-1795-0/minicluster-data/master-0/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786240941968412-1795-0/minicluster-data/master-0/wal
uuid: "3bc8638db8e5452a8dd40ac441c2e4c2"
format_stamp: "Formatted at 2026-08-09 02:05:16 on dist-test-slave-ss4n"
server_key: "75b2d6dd6abf5abc3c2fa52e41238e5f"
server_key_iv: "6cf0d8437d9d796d6afe28f097ec5a2a"
server_key_version: "encryptionkey@0"
I20260809 02:05:16.929628 16698 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786240941968412-1795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786240941968412-1795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786240941968412-1795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:05:16.943517 16698 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:05:16.947554 16698 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:05:16.947805 16698 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:05:16.956765 16698 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.254:33695
I20260809 02:05:16.956801 16768 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.254:33695 every 8 connection(s)
I20260809 02:05:16.957924 16698 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
I20260809 02:05:16.960789 16769 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:05:16.964754 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 16698
I20260809 02:05:16.964898 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786240941968412-1795-0/minicluster-data/master-0/wal/instance
I20260809 02:05:16.965240 1795 external_mini_cluster.cc:1428] Setting key 5f98fcf74095709616058f046b09a475
I20260809 02:05:16.967173 16769 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3bc8638db8e5452a8dd40ac441c2e4c2: Bootstrap starting.
I20260809 02:05:16.969424 16769 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3bc8638db8e5452a8dd40ac441c2e4c2: Neither blocks nor log segments found. Creating new log.
I20260809 02:05:16.970234 16769 log.cc:824] T 00000000000000000000000000000000 P 3bc8638db8e5452a8dd40ac441c2e4c2: Log is configured to *not* fsync() on all Append() calls
I20260809 02:05:16.972232 16769 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3bc8638db8e5452a8dd40ac441c2e4c2: No bootstrap required, opened a new log
Aug 09 02:05:16 dist-test-slave-ss4n krb5kdc[16685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241116, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.192.254@KRBTEST.COM
I20260809 02:05:16.976186 16769 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3bc8638db8e5452a8dd40ac441c2e4c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bc8638db8e5452a8dd40ac441c2e4c2" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 33695 } }
I20260809 02:05:16.976493 16769 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3bc8638db8e5452a8dd40ac441c2e4c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:05:16.976634 16769 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3bc8638db8e5452a8dd40ac441c2e4c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bc8638db8e5452a8dd40ac441c2e4c2, State: Initialized, Role: FOLLOWER
I20260809 02:05:16.977172 16769 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3bc8638db8e5452a8dd40ac441c2e4c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bc8638db8e5452a8dd40ac441c2e4c2" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 33695 } }
I20260809 02:05:16.977378 16769 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3bc8638db8e5452a8dd40ac441c2e4c2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:05:16.977476 16769 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3bc8638db8e5452a8dd40ac441c2e4c2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:05:16.977600 16769 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3bc8638db8e5452a8dd40ac441c2e4c2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:05:16.978554 16769 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3bc8638db8e5452a8dd40ac441c2e4c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bc8638db8e5452a8dd40ac441c2e4c2" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 33695 } }
I20260809 02:05:16.978951 16769 leader_election.cc:302] T 00000000000000000000000000000000 P 3bc8638db8e5452a8dd40ac441c2e4c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3bc8638db8e5452a8dd40ac441c2e4c2; no voters:
I20260809 02:05:16.979367 16769 leader_election.cc:288] T 00000000000000000000000000000000 P 3bc8638db8e5452a8dd40ac441c2e4c2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:05:16.979535 16774 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3bc8638db8e5452a8dd40ac441c2e4c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:05:16.979825 16774 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3bc8638db8e5452a8dd40ac441c2e4c2 [term 1 LEADER]: Becoming Leader. State: Replica: 3bc8638db8e5452a8dd40ac441c2e4c2, State: Running, Role: LEADER
I20260809 02:05:16.980273 16774 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3bc8638db8e5452a8dd40ac441c2e4c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bc8638db8e5452a8dd40ac441c2e4c2" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 33695 } }
I20260809 02:05:16.980726 16769 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3bc8638db8e5452a8dd40ac441c2e4c2 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:05:16.982108 16775 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3bc8638db8e5452a8dd40ac441c2e4c2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3bc8638db8e5452a8dd40ac441c2e4c2. Latest consensus state: current_term: 1 leader_uuid: "3bc8638db8e5452a8dd40ac441c2e4c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bc8638db8e5452a8dd40ac441c2e4c2" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 33695 } } }
I20260809 02:05:16.982251 16775 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3bc8638db8e5452a8dd40ac441c2e4c2 [sys.catalog]: This master's current role is: LEADER
I20260809 02:05:16.982384 16772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:16.966776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52094 (local address 127.1.192.254:33695)
0809 02:05:16.967293 (+ 517us) server_negotiation.cc:206] Beginning negotiation
0809 02:05:16.967316 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0809 02:05:16.967370 (+ 54us) server_negotiation.cc:407] Connection header received
0809 02:05:16.968117 (+ 747us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:05:16.968137 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:05:16.968492 (+ 355us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:05:16.968857 (+ 365us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:05:16.969886 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:16.971315 (+ 1429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:16.972278 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:16.972859 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:16.973059 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:05:16.975968 (+ 2909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:05:16.975974 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:05:16.975991 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:05:16.976024 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:05:16.978280 (+ 2256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:05:16.978834 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:05:16.978840 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:05:16.978846 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:05:16.978992 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:05:16.979345 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:05:16.979348 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:05:16.979350 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:05:16.979920 (+ 570us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:05:16.980117 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0809 02:05:16.980932 (+ 815us) server_negotiation.cc:299] Negotiation successful
0809 02:05:16.981263 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":161,"threads_started":1}
I20260809 02:05:16.982875 16776 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3bc8638db8e5452a8dd40ac441c2e4c2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3bc8638db8e5452a8dd40ac441c2e4c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bc8638db8e5452a8dd40ac441c2e4c2" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 33695 } } }
I20260809 02:05:16.983306 16776 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3bc8638db8e5452a8dd40ac441c2e4c2 [sys.catalog]: This master's current role is: LEADER
I20260809 02:05:16.983753 16784 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:05:16.986128 1795 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 02:05:16.987730 16784 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:05:16.993854 16784 catalog_manager.cc:1347] Generated new cluster ID: 8c5c011e7e3e4312acefedf1a441c2fd
I20260809 02:05:16.993945 16784 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:05:17.007027 16784 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:05:17.008256 16784 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:05:17.016330 16784 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3bc8638db8e5452a8dd40ac441c2e4c2: Generated new TSK 0
I20260809 02:05:17.017151 16784 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:05:17.042778 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 16698
2026-08-09T02:05:17Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1575 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:05:17 dist-test-slave-ss4n krb5kdc[16799](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:05:17 dist-test-slave-ss4n krb5kdc[16799](info): set up 2 sockets
Aug 09 02:05:17 dist-test-slave-ss4n krb5kdc[16799](info): commencing operation
krb5kdc: starting...
W20260809 02:05:18.205466 1795 mini_kdc.cc:121] Time spent starting KDC: real 1.140s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/krb5kdc/test-user.keytab.
Aug 09 02:05:18 dist-test-slave-ss4n krb5kdc[16799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241118, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:05:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:05:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.192.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.192.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.192.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab.
Entry for principal kudu/127.1.192.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab.
I20260809 02:05:18.350797 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.192.254:39909
--webserver_interface=127.1.192.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab
--principal=kudu/127.1.192.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:39143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.192.254:39909
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:05:18.471716 16812 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:05:18.472005 16812 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:05:18.472046 16812 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:05:18.476240 16812 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:05:18.476325 16812 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:05:18.476346 16812 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:05:18.476363 16812 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:05:18.476379 16812 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:05:18.482088 16812 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:39143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.192.254:39909
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab
--principal=kudu/127.1.192.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.192.254:39909
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.192.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16812
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:05:18.483265 16812 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:05:18.484303 16812 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:05:18.493474 16819 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:05:18.493430 16818 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:05:18.493430 16821 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:05:18.493742 16812 server_base.cc:1034] running on GCE node
I20260809 02:05:18.494472 16812 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:05:18.495733 16812 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:05:18.496949 16812 hybrid_clock.cc:648] HybridClock initialized: now 1786241118496928 us; error 36 us; skew 500 ppm
Aug 09 02:05:18 dist-test-slave-ss4n krb5kdc[16799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241118, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:05:18.500032 16812 init.cc:377] Logged in from keytab as kudu/127.1.192.254@KRBTEST.COM (short username kudu)
I20260809 02:05:18.501062 16812 webserver.cc:460] Webserver started at http://127.1.192.254:46177/ using document root <none> and password file <none>
I20260809 02:05:18.501754 16812 fs_manager.cc:359] Metadata directory not provided
I20260809 02:05:18.501855 16812 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:05:18.502091 16812 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:05:18.504254 16812 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/master-0/data/instance:
uuid: "a63f7b9159694eeebd13d63fa56373e3"
format_stamp: "Formatted at 2026-08-09 02:05:18 on dist-test-slave-ss4n"
server_key: "daad3b3cebb0f8a9e1960755b53597af"
server_key_iv: "e5de2a39794e0dffe6a648b932fb4255"
server_key_version: "encryptionkey@0"
I20260809 02:05:18.504854 16812 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/master-0/wal/instance:
uuid: "a63f7b9159694eeebd13d63fa56373e3"
format_stamp: "Formatted at 2026-08-09 02:05:18 on dist-test-slave-ss4n"
server_key: "daad3b3cebb0f8a9e1960755b53597af"
server_key_iv: "e5de2a39794e0dffe6a648b932fb4255"
server_key_version: "encryptionkey@0"
I20260809 02:05:18.508977 16812 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260809 02:05:18.511720 16828 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:05:18.513032 16812 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 02:05:18.513214 16812 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/master-0/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/master-0/wal
uuid: "a63f7b9159694eeebd13d63fa56373e3"
format_stamp: "Formatted at 2026-08-09 02:05:18 on dist-test-slave-ss4n"
server_key: "daad3b3cebb0f8a9e1960755b53597af"
server_key_iv: "e5de2a39794e0dffe6a648b932fb4255"
server_key_version: "encryptionkey@0"
I20260809 02:05:18.513350 16812 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:05:18.546277 16812 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:05:18.550170 16812 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:05:18.550412 16812 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:05:18.558105 16812 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.254:39909
I20260809 02:05:18.558404 16887 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.254:39909 every 8 connection(s)
I20260809 02:05:18.559232 16812 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
I20260809 02:05:18.562255 16888 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:05:18.568219 16888 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a63f7b9159694eeebd13d63fa56373e3: Bootstrap starting.
I20260809 02:05:18.569566 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 16812
I20260809 02:05:18.569706 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/master-0/wal/instance
I20260809 02:05:18.570017 1795 external_mini_cluster.cc:1428] Setting key f0871116c19ad283cbbc2d7f9f1fbd85
I20260809 02:05:18.571045 16888 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a63f7b9159694eeebd13d63fa56373e3: Neither blocks nor log segments found. Creating new log.
I20260809 02:05:18.571907 16888 log.cc:824] T 00000000000000000000000000000000 P a63f7b9159694eeebd13d63fa56373e3: Log is configured to *not* fsync() on all Append() calls
I20260809 02:05:18.573958 16888 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a63f7b9159694eeebd13d63fa56373e3: No bootstrap required, opened a new log
I20260809 02:05:18.577407 16888 raft_consensus.cc:348] T 00000000000000000000000000000000 P a63f7b9159694eeebd13d63fa56373e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a63f7b9159694eeebd13d63fa56373e3" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 39909 } }
I20260809 02:05:18.577683 16888 raft_consensus.cc:374] T 00000000000000000000000000000000 P a63f7b9159694eeebd13d63fa56373e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:05:18.577776 16888 raft_consensus.cc:729] T 00000000000000000000000000000000 P a63f7b9159694eeebd13d63fa56373e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a63f7b9159694eeebd13d63fa56373e3, State: Initialized, Role: FOLLOWER
I20260809 02:05:18.578264 16888 consensus_queue.cc:260] T 00000000000000000000000000000000 P a63f7b9159694eeebd13d63fa56373e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a63f7b9159694eeebd13d63fa56373e3" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 39909 } }
I20260809 02:05:18.578421 16888 raft_consensus.cc:388] T 00000000000000000000000000000000 P a63f7b9159694eeebd13d63fa56373e3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:05:18.578512 16888 raft_consensus.cc:482] T 00000000000000000000000000000000 P a63f7b9159694eeebd13d63fa56373e3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:05:18.578622 16888 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a63f7b9159694eeebd13d63fa56373e3 [term 0 FOLLOWER]: Advancing to term 1
Aug 09 02:05:18 dist-test-slave-ss4n krb5kdc[16799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241118, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.192.254@KRBTEST.COM
I20260809 02:05:18.579653 16888 raft_consensus.cc:504] T 00000000000000000000000000000000 P a63f7b9159694eeebd13d63fa56373e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a63f7b9159694eeebd13d63fa56373e3" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 39909 } }
I20260809 02:05:18.579998 16888 leader_election.cc:302] T 00000000000000000000000000000000 P a63f7b9159694eeebd13d63fa56373e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a63f7b9159694eeebd13d63fa56373e3; no voters:
I20260809 02:05:18.580330 16888 leader_election.cc:288] T 00000000000000000000000000000000 P a63f7b9159694eeebd13d63fa56373e3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:05:18.580709 16895 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a63f7b9159694eeebd13d63fa56373e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:05:18.580996 16895 raft_consensus.cc:686] T 00000000000000000000000000000000 P a63f7b9159694eeebd13d63fa56373e3 [term 1 LEADER]: Becoming Leader. State: Replica: a63f7b9159694eeebd13d63fa56373e3, State: Running, Role: LEADER
I20260809 02:05:18.581358 16895 consensus_queue.cc:237] T 00000000000000000000000000000000 P a63f7b9159694eeebd13d63fa56373e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a63f7b9159694eeebd13d63fa56373e3" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 39909 } }
I20260809 02:05:18.581728 16888 sys_catalog.cc:564] T 00000000000000000000000000000000 P a63f7b9159694eeebd13d63fa56373e3 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:05:18.584908 16896 sys_catalog.cc:455] T 00000000000000000000000000000000 P a63f7b9159694eeebd13d63fa56373e3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a63f7b9159694eeebd13d63fa56373e3. Latest consensus state: current_term: 1 leader_uuid: "a63f7b9159694eeebd13d63fa56373e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a63f7b9159694eeebd13d63fa56373e3" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 39909 } } }
I20260809 02:05:18.585369 16896 sys_catalog.cc:458] T 00000000000000000000000000000000 P a63f7b9159694eeebd13d63fa56373e3 [sys.catalog]: This master's current role is: LEADER
I20260809 02:05:18.584928 16893 sys_catalog.cc:455] T 00000000000000000000000000000000 P a63f7b9159694eeebd13d63fa56373e3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a63f7b9159694eeebd13d63fa56373e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a63f7b9159694eeebd13d63fa56373e3" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 39909 } } }
I20260809 02:05:18.585659 16893 sys_catalog.cc:458] T 00000000000000000000000000000000 P a63f7b9159694eeebd13d63fa56373e3 [sys.catalog]: This master's current role is: LEADER
I20260809 02:05:18.585841 16891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:18.571223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54648 (local address 127.1.192.254:39909)
0809 02:05:18.571735 (+ 512us) server_negotiation.cc:206] Beginning negotiation
0809 02:05:18.571751 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 02:05:18.571786 (+ 35us) server_negotiation.cc:407] Connection header received
0809 02:05:18.572726 (+ 940us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:05:18.572747 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:05:18.573070 (+ 323us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:05:18.573409 (+ 339us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:05:18.574325 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:18.575301 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:18.576170 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:18.576673 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:18.576849 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:05:18.579668 (+ 2819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:05:18.579674 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:05:18.579687 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:05:18.579715 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:05:18.581837 (+ 2122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:05:18.582344 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:05:18.582350 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:05:18.582356 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:05:18.582505 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:05:18.582851 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:05:18.582854 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:05:18.582856 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:05:18.583372 (+ 516us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:05:18.583585 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0809 02:05:18.584509 (+ 924us) server_negotiation.cc:299] Negotiation successful
0809 02:05:18.584828 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":174,"threads_started":1}
W20260809 02:05:18.587062 16912 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a63f7b9159694eeebd13d63fa56373e3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 02:05:18.587145 16912 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 02:05:18.587237 16915 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:05:18.588137 16915 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:05:18.593928 16915 catalog_manager.cc:1347] Generated new cluster ID: 83f659d0a92b496f90e277950880f819
I20260809 02:05:18.593997 16915 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:05:18.602921 16915 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:05:18.604266 16915 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:05:18.612885 16915 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a63f7b9159694eeebd13d63fa56373e3: Generated new TSK 0
I20260809 02:05:18.613801 16915 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.192.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.192.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.192.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.193.keytab.
Entry for principal kudu/127.1.192.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.193.keytab.
I20260809 02:05:18.657745 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.192.193:0
--local_ip_for_outbound_sockets=127.1.192.193
--webserver_interface=127.1.192.193
--webserver_port=0
--tserver_master_addrs=127.1.192.254:39909
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.193.keytab
--principal=kudu/127.1.192.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:39143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:05:18.785235 16919 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:05:18.785615 16919 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:05:18.785727 16919 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:05:18.790499 16919 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:05:18.790606 16919 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:05:18.790714 16919 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.192.193
I20260809 02:05:18.796841 16919 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:39143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.193.keytab
--principal=kudu/127.1.192.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.192.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.192.193
--webserver_port=0
--tserver_master_addrs=127.1.192.254:39909
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16919
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.192.193
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:05:18.798105 16919 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:05:18.799229 16919 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:05:18.810119 16928 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:05:18.810104 16926 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:05:18.810119 16925 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:05:18.810550 16919 server_base.cc:1034] running on GCE node
I20260809 02:05:18.811053 16919 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:05:18.811748 16919 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:05:18.812978 16919 hybrid_clock.cc:648] HybridClock initialized: now 1786241118812880 us; error 104 us; skew 500 ppm
Aug 09 02:05:18 dist-test-slave-ss4n krb5kdc[16799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241118, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:05:18.816448 16919 init.cc:377] Logged in from keytab as kudu/127.1.192.193@KRBTEST.COM (short username kudu)
I20260809 02:05:18.817709 16919 webserver.cc:460] Webserver started at http://127.1.192.193:36399/ using document root <none> and password file <none>
I20260809 02:05:18.818421 16919 fs_manager.cc:359] Metadata directory not provided
I20260809 02:05:18.818533 16919 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:05:18.818795 16919 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:05:18.821429 16919 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-0/data/instance:
uuid: "78569db4f20d4cfe884e53818df99efc"
format_stamp: "Formatted at 2026-08-09 02:05:18 on dist-test-slave-ss4n"
server_key: "ffa20ede60e9219240583a07f2b418c8"
server_key_iv: "2afde7e05377b354a77453b2d285e375"
server_key_version: "encryptionkey@0"
I20260809 02:05:18.822327 16919 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-0/wal/instance:
uuid: "78569db4f20d4cfe884e53818df99efc"
format_stamp: "Formatted at 2026-08-09 02:05:18 on dist-test-slave-ss4n"
server_key: "ffa20ede60e9219240583a07f2b418c8"
server_key_iv: "2afde7e05377b354a77453b2d285e375"
server_key_version: "encryptionkey@0"
I20260809 02:05:18.830111 16919 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.000s
I20260809 02:05:18.832955 16936 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:05:18.834379 16919 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 02:05:18.834553 16919 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-0/wal
uuid: "78569db4f20d4cfe884e53818df99efc"
format_stamp: "Formatted at 2026-08-09 02:05:18 on dist-test-slave-ss4n"
server_key: "ffa20ede60e9219240583a07f2b418c8"
server_key_iv: "2afde7e05377b354a77453b2d285e375"
server_key_version: "encryptionkey@0"
I20260809 02:05:18.834690 16919 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:05:18.848531 16919 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:05:18.851881 16919 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:05:18.852141 16919 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:05:18.852807 16919 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:05:18.853736 16919 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:05:18.853808 16919 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:05:18.853881 16919 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:05:18.853926 16919 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:05:18.863945 16919 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.193:44473
I20260809 02:05:18.863965 17060 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.193:44473 every 8 connection(s)
I20260809 02:05:18.864928 16919 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-0/data/info.pb
I20260809 02:05:18.869709 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 16919
I20260809 02:05:18.869865 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-0/wal/instance
I20260809 02:05:18.870220 1795 external_mini_cluster.cc:1428] Setting key d58824f44ac30bb86a72102dd89e32e2
Aug 09 02:05:18 dist-test-slave-ss4n krb5kdc[16799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241118, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.193@KRBTEST.COM for kudu/127.1.192.254@KRBTEST.COM
I20260809 02:05:18.880666 16891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:18.867145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.193:49533 (local address 127.1.192.254:39909)
0809 02:05:18.867327 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 02:05:18.867330 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:05:18.868435 (+ 1105us) server_negotiation.cc:407] Connection header received
0809 02:05:18.869344 (+ 909us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:05:18.869347 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:05:18.869397 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:05:18.869590 (+ 193us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:05:18.871165 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:18.871853 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:18.873096 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:18.873417 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:18.873600 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:05:18.876736 (+ 3136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:05:18.876739 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:05:18.876742 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:05:18.876773 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:05:18.878459 (+ 1686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:05:18.879036 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:05:18.879040 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:05:18.879042 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:05:18.879132 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:05:18.879509 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:05:18.879513 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:05:18.879514 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:05:18.879717 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:05:18.879827 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 02:05:18.880358 (+ 531us) server_negotiation.cc:299] Negotiation successful
0809 02:05:18.880492 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
WARNING: no policy specified for kudu/127.1.192.194@KRBTEST.COM; defaulting to no policy
I20260809 02:05:18.881722 17063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:18.867500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:39909 (local address 127.1.192.193:49533)
0809 02:05:18.868272 (+ 772us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:18.868316 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 02:05:18.869133 (+ 817us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:05:18.869796 (+ 663us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:05:18.869808 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:05:18.870347 (+ 539us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:05:18.870946 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:05:18.870961 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:05:18.871991 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:05:18.871997 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:05:18.872943 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:05:18.872953 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:05:18.873663 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:05:18.873667 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:05:18.873845 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:05:18.874583 (+ 738us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:05:18.874611 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:05:18.876535 (+ 1924us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:05:18.878599 (+ 2064us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:05:18.878606 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:05:18.878624 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:05:18.878916 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:05:18.879234 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:05:18.879238 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:05:18.879240 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:05:18.879398 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:05:18.879828 (+ 430us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:05:18.879837 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:05:18.880115 (+ 278us) client_negotiation.cc:769] Sending connection context
0809 02:05:18.880317 (+ 202us) client_negotiation.cc:240] Negotiation successful
0809 02:05:18.880689 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":526,"thread_start_us":125,"threads_started":1}
I20260809 02:05:18.883301 17061 heartbeater.cc:347] Connected to a master server at 127.1.192.254:39909
I20260809 02:05:18.883591 17061 heartbeater.cc:464] Registering TS with master...
I20260809 02:05:18.884187 17061 heartbeater.cc:511] Master 127.1.192.254:39909 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.192.194@KRBTEST.COM" created.
I20260809 02:05:18.886157 16847 ts_manager.cc:194] Registered new tserver with Master: 78569db4f20d4cfe884e53818df99efc (127.1.192.193:44473)
I20260809 02:05:18.887420 16847 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.192.193@KRBTEST.COM'} at 127.1.192.193:49533
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.192.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.194.keytab.
Entry for principal kudu/127.1.192.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.194.keytab.
I20260809 02:05:18.906987 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.192.194:0
--local_ip_for_outbound_sockets=127.1.192.194
--webserver_interface=127.1.192.194
--webserver_port=0
--tserver_master_addrs=127.1.192.254:39909
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.194.keytab
--principal=kudu/127.1.192.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:39143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:05:19.025786 17066 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:05:19.026062 17066 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:05:19.026106 17066 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:05:19.030136 17066 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:05:19.030203 17066 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:05:19.030263 17066 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.192.194
I20260809 02:05:19.035713 17066 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:39143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.194.keytab
--principal=kudu/127.1.192.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.192.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.192.194
--webserver_port=0
--tserver_master_addrs=127.1.192.254:39909
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17066
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.192.194
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:05:19.036792 17066 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:05:19.037904 17066 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:05:19.048247 17071 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:05:19.048300 17066 server_base.cc:1034] running on GCE node
W20260809 02:05:19.048277 17072 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:05:19.048240 17074 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:05:19.049042 17066 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:05:19.049839 17066 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:05:19.051081 17066 hybrid_clock.cc:648] HybridClock initialized: now 1786241119051062 us; error 30 us; skew 500 ppm
Aug 09 02:05:19 dist-test-slave-ss4n krb5kdc[16799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241119, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:05:19.054481 17066 init.cc:377] Logged in from keytab as kudu/127.1.192.194@KRBTEST.COM (short username kudu)
I20260809 02:05:19.055519 17066 webserver.cc:460] Webserver started at http://127.1.192.194:42009/ using document root <none> and password file <none>
I20260809 02:05:19.056124 17066 fs_manager.cc:359] Metadata directory not provided
I20260809 02:05:19.056222 17066 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:05:19.056464 17066 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:05:19.058665 17066 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-1/data/instance:
uuid: "ce876b1fc4a840588948104f4e7eda5d"
format_stamp: "Formatted at 2026-08-09 02:05:19 on dist-test-slave-ss4n"
server_key: "62739a957f5623802f673291825320ae"
server_key_iv: "22ee1ca0fb96c18830f71b6ffffcfede"
server_key_version: "encryptionkey@0"
I20260809 02:05:19.059266 17066 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-1/wal/instance:
uuid: "ce876b1fc4a840588948104f4e7eda5d"
format_stamp: "Formatted at 2026-08-09 02:05:19 on dist-test-slave-ss4n"
server_key: "62739a957f5623802f673291825320ae"
server_key_iv: "22ee1ca0fb96c18830f71b6ffffcfede"
server_key_version: "encryptionkey@0"
I20260809 02:05:19.063148 17066 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260809 02:05:19.065825 17082 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:05:19.067111 17066 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 02:05:19.067260 17066 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-1/wal
uuid: "ce876b1fc4a840588948104f4e7eda5d"
format_stamp: "Formatted at 2026-08-09 02:05:19 on dist-test-slave-ss4n"
server_key: "62739a957f5623802f673291825320ae"
server_key_iv: "22ee1ca0fb96c18830f71b6ffffcfede"
server_key_version: "encryptionkey@0"
I20260809 02:05:19.067386 17066 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:05:19.081321 17066 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:05:19.085289 17066 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:05:19.085551 17066 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:05:19.086246 17066 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:05:19.087051 17066 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:05:19.087102 17066 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:05:19.087143 17066 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:05:19.087159 17066 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:05:19.096180 17066 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.194:38717
I20260809 02:05:19.096284 17204 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.194:38717 every 8 connection(s)
I20260809 02:05:19.097137 17066 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-1/data/info.pb
I20260809 02:05:19.105266 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 17066
I20260809 02:05:19.105376 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-1/wal/instance
I20260809 02:05:19.105729 1795 external_mini_cluster.cc:1428] Setting key 4859b0bf557c09aa054d18bba8790a84
Aug 09 02:05:19 dist-test-slave-ss4n krb5kdc[16799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241119, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.194@KRBTEST.COM for kudu/127.1.192.254@KRBTEST.COM
I20260809 02:05:19.113019 16891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:19.099256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.194:34437 (local address 127.1.192.254:39909)
0809 02:05:19.099410 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0809 02:05:19.099414 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:05:19.100573 (+ 1159us) server_negotiation.cc:407] Connection header received
0809 02:05:19.101668 (+ 1095us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:05:19.101671 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:05:19.101735 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:05:19.101839 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:05:19.103130 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:19.103755 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:19.105017 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:19.105280 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:19.105420 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:05:19.108464 (+ 3044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:05:19.108469 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:05:19.108471 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:05:19.108508 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:05:19.110497 (+ 1989us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:05:19.111184 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:05:19.111187 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:05:19.111189 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:05:19.111272 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:05:19.111697 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:05:19.111700 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:05:19.111701 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:05:19.111893 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:05:19.111990 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0809 02:05:19.112714 (+ 724us) server_negotiation.cc:299] Negotiation successful
0809 02:05:19.112866 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 02:05:19.114249 17208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:19.099690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:39909 (local address 127.1.192.194:34437)
0809 02:05:19.100405 (+ 715us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:19.100449 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 02:05:19.101318 (+ 869us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:05:19.102023 (+ 705us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:05:19.102036 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:05:19.102453 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:05:19.102956 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:05:19.102970 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:05:19.103900 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:05:19.103910 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:05:19.104877 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:05:19.104887 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:05:19.105547 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:05:19.105555 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:05:19.105729 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:05:19.106509 (+ 780us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:05:19.106532 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:05:19.108283 (+ 1751us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:05:19.110676 (+ 2393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:05:19.110681 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:05:19.110695 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:05:19.111075 (+ 380us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:05:19.111371 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:05:19.111377 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:05:19.111382 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:05:19.111575 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:05:19.112006 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:05:19.112016 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:05:19.112393 (+ 377us) client_negotiation.cc:769] Sending connection context
0809 02:05:19.112648 (+ 255us) client_negotiation.cc:240] Negotiation successful
0809 02:05:19.113056 (+ 408us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":520,"thread_start_us":123,"threads_started":1}
I20260809 02:05:19.115763 17205 heartbeater.cc:347] Connected to a master server at 127.1.192.254:39909
I20260809 02:05:19.116046 17205 heartbeater.cc:464] Registering TS with master...
I20260809 02:05:19.116708 17205 heartbeater.cc:511] Master 127.1.192.254:39909 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.1.192.195@KRBTEST.COM; defaulting to no policy
I20260809 02:05:19.118151 16847 ts_manager.cc:194] Registered new tserver with Master: ce876b1fc4a840588948104f4e7eda5d (127.1.192.194:38717)
I20260809 02:05:19.118898 16847 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.192.194@KRBTEST.COM'} at 127.1.192.194:34437
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.192.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.192.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.195.keytab.
Entry for principal kudu/127.1.192.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.195.keytab.
I20260809 02:05:19.141685 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.192.195:0
--local_ip_for_outbound_sockets=127.1.192.195
--webserver_interface=127.1.192.195
--webserver_port=0
--tserver_master_addrs=127.1.192.254:39909
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.195.keytab
--principal=kudu/127.1.192.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:39143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:05:19.256583 17211 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:05:19.256961 17211 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:05:19.257041 17211 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:05:19.261402 17211 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:05:19.261544 17211 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:05:19.261646 17211 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.192.195
I20260809 02:05:19.267313 17211 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:39143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.195.keytab
--principal=kudu/127.1.192.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.192.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.192.195
--webserver_port=0
--tserver_master_addrs=127.1.192.254:39909
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17211
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.192.195
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:05:19.268479 17211 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:05:19.269577 17211 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:05:19.279983 17217 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:05:19.280104 17219 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:05:19.280043 17211 server_base.cc:1034] running on GCE node
W20260809 02:05:19.279985 17216 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:05:19.280616 17211 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:05:19.281332 17211 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:05:19.282521 17211 hybrid_clock.cc:648] HybridClock initialized: now 1786241119282486 us; error 35 us; skew 500 ppm
Aug 09 02:05:19 dist-test-slave-ss4n krb5kdc[16799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241119, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:05:19.285583 17211 init.cc:377] Logged in from keytab as kudu/127.1.192.195@KRBTEST.COM (short username kudu)
I20260809 02:05:19.286487 17211 webserver.cc:460] Webserver started at http://127.1.192.195:46035/ using document root <none> and password file <none>
I20260809 02:05:19.287014 17211 fs_manager.cc:359] Metadata directory not provided
I20260809 02:05:19.287082 17211 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:05:19.287251 17211 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:05:19.289297 17211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-2/data/instance:
uuid: "948a4d8653d148859d72dea2948ebb96"
format_stamp: "Formatted at 2026-08-09 02:05:19 on dist-test-slave-ss4n"
server_key: "644cd5db26089d8683d1c9b51441edd2"
server_key_iv: "b410d2bded5c3f09667fec9df33a7206"
server_key_version: "encryptionkey@0"
I20260809 02:05:19.289963 17211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-2/wal/instance:
uuid: "948a4d8653d148859d72dea2948ebb96"
format_stamp: "Formatted at 2026-08-09 02:05:19 on dist-test-slave-ss4n"
server_key: "644cd5db26089d8683d1c9b51441edd2"
server_key_iv: "b410d2bded5c3f09667fec9df33a7206"
server_key_version: "encryptionkey@0"
I20260809 02:05:19.293819 17211 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260809 02:05:19.296413 17227 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:05:19.298095 17211 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 02:05:19.298269 17211 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-2/wal
uuid: "948a4d8653d148859d72dea2948ebb96"
format_stamp: "Formatted at 2026-08-09 02:05:19 on dist-test-slave-ss4n"
server_key: "644cd5db26089d8683d1c9b51441edd2"
server_key_iv: "b410d2bded5c3f09667fec9df33a7206"
server_key_version: "encryptionkey@0"
I20260809 02:05:19.298411 17211 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:05:19.318929 17211 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:05:19.322980 17211 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:05:19.323231 17211 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:05:19.323921 17211 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:05:19.324898 17211 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:05:19.324977 17211 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:05:19.325059 17211 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:05:19.325124 17211 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:05:19.335620 17211 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.195:38931
I20260809 02:05:19.335661 17351 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.195:38931 every 8 connection(s)
I20260809 02:05:19.336650 17211 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-2/data/info.pb
I20260809 02:05:19.339387 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 17211
I20260809 02:05:19.339547 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-2/wal/instance
I20260809 02:05:19.339888 1795 external_mini_cluster.cc:1428] Setting key 4e66fff10c22b7aca9fbe39f3e6bc7f8
Aug 09 02:05:19 dist-test-slave-ss4n krb5kdc[16799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241119, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.195@KRBTEST.COM for kudu/127.1.192.254@KRBTEST.COM
I20260809 02:05:19.351339 16891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:19.338772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.195:40245 (local address 127.1.192.254:39909)
0809 02:05:19.338950 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0809 02:05:19.338954 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:05:19.340053 (+ 1099us) server_negotiation.cc:407] Connection header received
0809 02:05:19.341163 (+ 1110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:05:19.341166 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:05:19.341222 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:05:19.341302 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:05:19.342661 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:19.343204 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:19.344390 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:19.344621 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:19.344734 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:05:19.347597 (+ 2863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:05:19.347601 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:05:19.347604 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:05:19.347635 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:05:19.349132 (+ 1497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:05:19.349741 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:05:19.349745 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:05:19.349747 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:05:19.349834 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:05:19.350202 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:05:19.350205 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:05:19.350207 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:05:19.350399 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:05:19.350508 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 02:05:19.351028 (+ 520us) server_negotiation.cc:299] Negotiation successful
0809 02:05:19.351163 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 02:05:19.352187 17354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:19.339083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:39909 (local address 127.1.192.195:40245)
0809 02:05:19.339880 (+ 797us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:19.339936 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0809 02:05:19.340931 (+ 995us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:05:19.341495 (+ 564us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:05:19.341505 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:05:19.341923 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:05:19.342520 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:05:19.342534 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:05:19.343334 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:05:19.343338 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:05:19.344256 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:05:19.344266 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:05:19.344780 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:05:19.344784 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:05:19.344957 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:05:19.345754 (+ 797us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:05:19.345770 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:05:19.347405 (+ 1635us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:05:19.349266 (+ 1861us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:05:19.349272 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:05:19.349288 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:05:19.349635 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:05:19.349931 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:05:19.349934 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:05:19.349937 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:05:19.350097 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:05:19.350508 (+ 411us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:05:19.350516 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:05:19.350781 (+ 265us) client_negotiation.cc:769] Sending connection context
0809 02:05:19.350972 (+ 191us) client_negotiation.cc:240] Negotiation successful
0809 02:05:19.351314 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":545,"thread_start_us":140,"threads_started":1}
I20260809 02:05:19.353607 17352 heartbeater.cc:347] Connected to a master server at 127.1.192.254:39909
I20260809 02:05:19.353873 17352 heartbeater.cc:464] Registering TS with master...
I20260809 02:05:19.354378 17352 heartbeater.cc:511] Master 127.1.192.254:39909 requested a full tablet report, sending...
I20260809 02:05:19.355468 16847 ts_manager.cc:194] Registered new tserver with Master: 948a4d8653d148859d72dea2948ebb96 (127.1.192.195:38931)
I20260809 02:05:19.356166 16847 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.192.195@KRBTEST.COM'} at 127.1.192.195:40245
I20260809 02:05:19.367033 1795 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 02:05:19.376868 16891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:19.369245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54652 (local address 127.1.192.254:39909)
0809 02:05:19.369421 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0809 02:05:19.369427 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:05:19.369640 (+ 213us) server_negotiation.cc:407] Connection header received
0809 02:05:19.369711 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:05:19.369713 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:05:19.369761 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:05:19.369832 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:05:19.370614 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:19.371213 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:19.372175 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:19.372379 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:19.372459 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:05:19.374036 (+ 1577us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:05:19.374039 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:05:19.374041 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:05:19.374064 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:05:19.375347 (+ 1283us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:05:19.375737 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:05:19.375740 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:05:19.375741 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:05:19.375815 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:05:19.376064 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:05:19.376066 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:05:19.376067 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:05:19.376271 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:05:19.376373 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 02:05:19.376568 (+ 195us) server_negotiation.cc:299] Negotiation successful
0809 02:05:19.376666 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 02:05:19.380419 16847 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54652:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 02:05:19.384202 16847 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:05:19.399602 17365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:19.393662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40564 (local address 127.1.192.195:38931)
0809 02:05:19.395008 (+ 1346us) server_negotiation.cc:206] Beginning negotiation
0809 02:05:19.395017 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 02:05:19.395038 (+ 21us) server_negotiation.cc:407] Connection header received
0809 02:05:19.395100 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:05:19.395104 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:05:19.395272 (+ 168us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:05:19.395436 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:05:19.395753 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:19.396583 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:19.398240 (+ 1657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:19.398835 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:19.398959 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:05:19.399154 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0809 02:05:19.399352 (+ 198us) server_negotiation.cc:299] Negotiation successful
0809 02:05:19.399452 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1256,"thread_start_us":148,"threads_started":1}
I20260809 02:05:19.399601 17364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:19.393382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.195:38931 (local address 127.0.0.1:40564)
0809 02:05:19.394060 (+ 678us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:19.394117 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0809 02:05:19.394269 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:05:19.395429 (+ 1160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:05:19.395435 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:05:19.395459 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:05:19.395617 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:05:19.395624 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:05:19.396764 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:05:19.396767 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:05:19.398113 (+ 1346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:05:19.398123 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:05:19.399015 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:05:19.399019 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:05:19.399127 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:05:19.399168 (+ 41us) client_negotiation.cc:769] Sending connection context
0809 02:05:19.399310 (+ 142us) client_negotiation.cc:240] Negotiation successful
0809 02:05:19.399440 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":479,"mutex_wait_us":60,"spinlock_wait_cycles":3072,"thread_start_us":68,"threads_started":1}
I20260809 02:05:19.400552 17363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:19.393648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.193:44473 (local address 127.0.0.1:58958)
0809 02:05:19.394073 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:19.394116 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 02:05:19.394269 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:05:19.395649 (+ 1380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:05:19.395653 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:05:19.395684 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:05:19.395794 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:05:19.395805 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:05:19.397451 (+ 1646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:05:19.397454 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:05:19.398869 (+ 1415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:05:19.398882 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:05:19.400155 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:05:19.400159 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:05:19.400247 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:05:19.400270 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 02:05:19.400354 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 02:05:19.400417 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":373,"mutex_wait_us":64,"spinlock_wait_cycles":12160,"thread_start_us":130,"threads_started":1}
I20260809 02:05:19.401065 17367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:19.394596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58958 (local address 127.1.192.193:44473)
0809 02:05:19.395223 (+ 627us) server_negotiation.cc:206] Beginning negotiation
0809 02:05:19.395230 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 02:05:19.395250 (+ 20us) server_negotiation.cc:407] Connection header received
0809 02:05:19.395318 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:05:19.395323 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:05:19.395480 (+ 157us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:05:19.395966 (+ 486us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:05:19.396129 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:19.397324 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:19.399052 (+ 1728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:19.399997 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:19.400124 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:05:19.400277 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0809 02:05:19.400826 (+ 549us) server_negotiation.cc:299] Negotiation successful
0809 02:05:19.400921 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":530,"thread_start_us":100,"threads_started":1}
I20260809 02:05:19.401521 17366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:19.394274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.194:38717 (local address 127.0.0.1:35022)
0809 02:05:19.394650 (+ 376us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:19.394664 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 02:05:19.394761 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:05:19.395621 (+ 860us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:05:19.395624 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:05:19.395640 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:05:19.395760 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:05:19.395766 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:05:19.397359 (+ 1593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:05:19.397366 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:05:19.398937 (+ 1571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:05:19.398949 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:05:19.400843 (+ 1894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:05:19.400847 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:05:19.400957 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:05:19.400982 (+ 25us) client_negotiation.cc:769] Sending connection context
0809 02:05:19.401067 (+ 85us) client_negotiation.cc:240] Negotiation successful
0809 02:05:19.401318 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":313,"mutex_wait_us":41,"spinlock_wait_cycles":3840,"thread_start_us":96,"threads_started":1}
I20260809 02:05:19.401737 17368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:19.394708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35022 (local address 127.1.192.194:38717)
0809 02:05:19.395117 (+ 409us) server_negotiation.cc:206] Beginning negotiation
0809 02:05:19.395131 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 02:05:19.395157 (+ 26us) server_negotiation.cc:407] Connection header received
0809 02:05:19.395242 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:05:19.395248 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:05:19.395450 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:05:19.395633 (+ 183us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:05:19.395854 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:19.397185 (+ 1331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:19.399083 (+ 1898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:19.400406 (+ 1323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:19.400548 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:05:19.401308 (+ 760us) server_negotiation.cc:1035] Waiting for connection context
0809 02:05:19.401367 (+ 59us) server_negotiation.cc:299] Negotiation successful
0809 02:05:19.401460 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":199,"threads_started":1}
I20260809 02:05:19.403028 17278 tablet_service.cc:1467] Processing CreateTablet for tablet d656709c8cdd43abbb50ca483cec67f2 (DEFAULT_TABLE table=test-table [id=cc1433a9f55942c79e618388f04df8a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:05:19.404260 17278 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d656709c8cdd43abbb50ca483cec67f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:05:19.403849 16989 tablet_service.cc:1467] Processing CreateTablet for tablet d656709c8cdd43abbb50ca483cec67f2 (DEFAULT_TABLE table=test-table [id=cc1433a9f55942c79e618388f04df8a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:05:19.404834 16989 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d656709c8cdd43abbb50ca483cec67f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:05:19.404410 17134 tablet_service.cc:1467] Processing CreateTablet for tablet d656709c8cdd43abbb50ca483cec67f2 (DEFAULT_TABLE table=test-table [id=cc1433a9f55942c79e618388f04df8a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:05:19.405465 17134 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d656709c8cdd43abbb50ca483cec67f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:05:19.412460 17371 tablet_bootstrap.cc:492] T d656709c8cdd43abbb50ca483cec67f2 P 948a4d8653d148859d72dea2948ebb96: Bootstrap starting.
I20260809 02:05:19.412771 17369 tablet_bootstrap.cc:492] T d656709c8cdd43abbb50ca483cec67f2 P ce876b1fc4a840588948104f4e7eda5d: Bootstrap starting.
I20260809 02:05:19.412029 17370 tablet_bootstrap.cc:492] T d656709c8cdd43abbb50ca483cec67f2 P 78569db4f20d4cfe884e53818df99efc: Bootstrap starting.
I20260809 02:05:19.415208 17370 tablet_bootstrap.cc:654] T d656709c8cdd43abbb50ca483cec67f2 P 78569db4f20d4cfe884e53818df99efc: Neither blocks nor log segments found. Creating new log.
I20260809 02:05:19.415676 17371 tablet_bootstrap.cc:654] T d656709c8cdd43abbb50ca483cec67f2 P 948a4d8653d148859d72dea2948ebb96: Neither blocks nor log segments found. Creating new log.
I20260809 02:05:19.415783 17369 tablet_bootstrap.cc:654] T d656709c8cdd43abbb50ca483cec67f2 P ce876b1fc4a840588948104f4e7eda5d: Neither blocks nor log segments found. Creating new log.
I20260809 02:05:19.416585 17369 log.cc:824] T d656709c8cdd43abbb50ca483cec67f2 P ce876b1fc4a840588948104f4e7eda5d: Log is configured to *not* fsync() on all Append() calls
I20260809 02:05:19.416666 17370 log.cc:824] T d656709c8cdd43abbb50ca483cec67f2 P 78569db4f20d4cfe884e53818df99efc: Log is configured to *not* fsync() on all Append() calls
I20260809 02:05:19.416718 17371 log.cc:824] T d656709c8cdd43abbb50ca483cec67f2 P 948a4d8653d148859d72dea2948ebb96: Log is configured to *not* fsync() on all Append() calls
I20260809 02:05:19.418862 17371 tablet_bootstrap.cc:492] T d656709c8cdd43abbb50ca483cec67f2 P 948a4d8653d148859d72dea2948ebb96: No bootstrap required, opened a new log
I20260809 02:05:19.418956 17370 tablet_bootstrap.cc:492] T d656709c8cdd43abbb50ca483cec67f2 P 78569db4f20d4cfe884e53818df99efc: No bootstrap required, opened a new log
I20260809 02:05:19.418956 17369 tablet_bootstrap.cc:492] T d656709c8cdd43abbb50ca483cec67f2 P ce876b1fc4a840588948104f4e7eda5d: No bootstrap required, opened a new log
I20260809 02:05:19.419122 17371 ts_tablet_manager.cc:1397] T d656709c8cdd43abbb50ca483cec67f2 P 948a4d8653d148859d72dea2948ebb96: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260809 02:05:19.419176 17369 ts_tablet_manager.cc:1397] T d656709c8cdd43abbb50ca483cec67f2 P ce876b1fc4a840588948104f4e7eda5d: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20260809 02:05:19.419176 17370 ts_tablet_manager.cc:1397] T d656709c8cdd43abbb50ca483cec67f2 P 78569db4f20d4cfe884e53818df99efc: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260809 02:05:19.422130 17371 raft_consensus.cc:348] T d656709c8cdd43abbb50ca483cec67f2 P 948a4d8653d148859d72dea2948ebb96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "948a4d8653d148859d72dea2948ebb96" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 38931 } } peers { permanent_uuid: "78569db4f20d4cfe884e53818df99efc" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 44473 } } peers { permanent_uuid: "ce876b1fc4a840588948104f4e7eda5d" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38717 } }
I20260809 02:05:19.422506 17371 raft_consensus.cc:374] T d656709c8cdd43abbb50ca483cec67f2 P 948a4d8653d148859d72dea2948ebb96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:05:19.422564 17371 raft_consensus.cc:729] T d656709c8cdd43abbb50ca483cec67f2 P 948a4d8653d148859d72dea2948ebb96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 948a4d8653d148859d72dea2948ebb96, State: Initialized, Role: FOLLOWER
I20260809 02:05:19.423039 17371 consensus_queue.cc:260] T d656709c8cdd43abbb50ca483cec67f2 P 948a4d8653d148859d72dea2948ebb96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "948a4d8653d148859d72dea2948ebb96" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 38931 } } peers { permanent_uuid: "78569db4f20d4cfe884e53818df99efc" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 44473 } } peers { permanent_uuid: "ce876b1fc4a840588948104f4e7eda5d" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38717 } }
I20260809 02:05:19.423923 17352 heartbeater.cc:503] Master 127.1.192.254:39909 was elected leader, sending a full tablet report...
I20260809 02:05:19.423753 17369 raft_consensus.cc:348] T d656709c8cdd43abbb50ca483cec67f2 P ce876b1fc4a840588948104f4e7eda5d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "948a4d8653d148859d72dea2948ebb96" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 38931 } } peers { permanent_uuid: "78569db4f20d4cfe884e53818df99efc" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 44473 } } peers { permanent_uuid: "ce876b1fc4a840588948104f4e7eda5d" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38717 } }
I20260809 02:05:19.424049 17369 raft_consensus.cc:374] T d656709c8cdd43abbb50ca483cec67f2 P ce876b1fc4a840588948104f4e7eda5d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:05:19.424113 17369 raft_consensus.cc:729] T d656709c8cdd43abbb50ca483cec67f2 P ce876b1fc4a840588948104f4e7eda5d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce876b1fc4a840588948104f4e7eda5d, State: Initialized, Role: FOLLOWER
I20260809 02:05:19.424127 17371 ts_tablet_manager.cc:1428] T d656709c8cdd43abbb50ca483cec67f2 P 948a4d8653d148859d72dea2948ebb96: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260809 02:05:19.424557 17369 consensus_queue.cc:260] T d656709c8cdd43abbb50ca483cec67f2 P ce876b1fc4a840588948104f4e7eda5d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "948a4d8653d148859d72dea2948ebb96" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 38931 } } peers { permanent_uuid: "78569db4f20d4cfe884e53818df99efc" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 44473 } } peers { permanent_uuid: "ce876b1fc4a840588948104f4e7eda5d" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38717 } }
I20260809 02:05:19.425323 17205 heartbeater.cc:503] Master 127.1.192.254:39909 was elected leader, sending a full tablet report...
I20260809 02:05:19.425637 17369 ts_tablet_manager.cc:1428] T d656709c8cdd43abbb50ca483cec67f2 P ce876b1fc4a840588948104f4e7eda5d: Time spent starting tablet: real 0.006s user 0.008s sys 0.000s
I20260809 02:05:19.425685 17370 raft_consensus.cc:348] T d656709c8cdd43abbb50ca483cec67f2 P 78569db4f20d4cfe884e53818df99efc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "948a4d8653d148859d72dea2948ebb96" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 38931 } } peers { permanent_uuid: "78569db4f20d4cfe884e53818df99efc" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 44473 } } peers { permanent_uuid: "ce876b1fc4a840588948104f4e7eda5d" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38717 } }
I20260809 02:05:19.425957 17370 raft_consensus.cc:374] T d656709c8cdd43abbb50ca483cec67f2 P 78569db4f20d4cfe884e53818df99efc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:05:19.426074 17370 raft_consensus.cc:729] T d656709c8cdd43abbb50ca483cec67f2 P 78569db4f20d4cfe884e53818df99efc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78569db4f20d4cfe884e53818df99efc, State: Initialized, Role: FOLLOWER
I20260809 02:05:19.426602 17370 consensus_queue.cc:260] T d656709c8cdd43abbb50ca483cec67f2 P 78569db4f20d4cfe884e53818df99efc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "948a4d8653d148859d72dea2948ebb96" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 38931 } } peers { permanent_uuid: "78569db4f20d4cfe884e53818df99efc" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 44473 } } peers { permanent_uuid: "ce876b1fc4a840588948104f4e7eda5d" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38717 } }
I20260809 02:05:19.427556 17061 heartbeater.cc:503] Master 127.1.192.254:39909 was elected leader, sending a full tablet report...
I20260809 02:05:19.428207 17370 ts_tablet_manager.cc:1428] T d656709c8cdd43abbb50ca483cec67f2 P 78569db4f20d4cfe884e53818df99efc: Time spent starting tablet: real 0.009s user 0.009s sys 0.000s
I20260809 02:05:19.466401 17375 raft_consensus.cc:482] T d656709c8cdd43abbb50ca483cec67f2 P 948a4d8653d148859d72dea2948ebb96 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:05:19.466661 17375 raft_consensus.cc:504] T d656709c8cdd43abbb50ca483cec67f2 P 948a4d8653d148859d72dea2948ebb96 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "948a4d8653d148859d72dea2948ebb96" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 38931 } } peers { permanent_uuid: "78569db4f20d4cfe884e53818df99efc" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 44473 } } peers { permanent_uuid: "ce876b1fc4a840588948104f4e7eda5d" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38717 } }
I20260809 02:05:19.467875 17375 leader_election.cc:288] T d656709c8cdd43abbb50ca483cec67f2 P 948a4d8653d148859d72dea2948ebb96 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 78569db4f20d4cfe884e53818df99efc (127.1.192.193:44473), ce876b1fc4a840588948104f4e7eda5d (127.1.192.194:38717)
I20260809 02:05:19.472162 17368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:19.468298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.195:56811 (local address 127.1.192.194:38717)
0809 02:05:19.468439 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0809 02:05:19.468444 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:05:19.468463 (+ 19us) server_negotiation.cc:407] Connection header received
0809 02:05:19.468667 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:05:19.468670 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:05:19.468738 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:05:19.468839 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:05:19.469119 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:19.469711 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:19.470960 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:19.471558 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:19.471693 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:05:19.471752 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 02:05:19.471954 (+ 202us) server_negotiation.cc:299] Negotiation successful
0809 02:05:19.472032 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 02:05:19.472159 17354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:19.468151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.194:38717 (local address 127.1.192.195:56811)
0809 02:05:19.468394 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:19.468413 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 02:05:19.468520 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:05:19.468850 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:05:19.468854 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:05:19.468873 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:05:19.468998 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:05:19.469004 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:05:19.469818 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:05:19.469824 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:05:19.470834 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:05:19.470843 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:05:19.471674 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:05:19.471678 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:05:19.471794 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:05:19.471820 (+ 26us) client_negotiation.cc:769] Sending connection context
0809 02:05:19.471907 (+ 87us) client_negotiation.cc:240] Negotiation successful
0809 02:05:19.471991 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"mutex_wait_us":1}
I20260809 02:05:19.472448 17378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:19.468212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.193:44473 (local address 127.1.192.195:49181)
0809 02:05:19.468840 (+ 628us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:19.468852 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:05:19.468979 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:05:19.469250 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:05:19.469253 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:05:19.469269 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:05:19.469399 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:05:19.469405 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:05:19.470187 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:05:19.470190 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:05:19.471161 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:05:19.471174 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:05:19.471989 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:05:19.471993 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:05:19.472110 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:05:19.472131 (+ 21us) client_negotiation.cc:769] Sending connection context
0809 02:05:19.472216 (+ 85us) client_negotiation.cc:240] Negotiation successful
0809 02:05:19.472285 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":560,"mutex_wait_us":21,"thread_start_us":146,"threads_started":1}
I20260809 02:05:19.472494 17367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:19.468487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.195:49181 (local address 127.1.192.193:44473)
0809 02:05:19.468642 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0809 02:05:19.468646 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:05:19.468938 (+ 292us) server_negotiation.cc:407] Connection header received
0809 02:05:19.469094 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:05:19.469097 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:05:19.469148 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:05:19.469240 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:05:19.469538 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:19.470029 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:19.471285 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:19.471870 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:19.471986 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:05:19.472031 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0809 02:05:19.472266 (+ 235us) server_negotiation.cc:299] Negotiation successful
0809 02:05:19.472360 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 02:05:19.473208 17010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d656709c8cdd43abbb50ca483cec67f2" candidate_uuid: "948a4d8653d148859d72dea2948ebb96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78569db4f20d4cfe884e53818df99efc" is_pre_election: true
I20260809 02:05:19.473306 17155 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d656709c8cdd43abbb50ca483cec67f2" candidate_uuid: "948a4d8653d148859d72dea2948ebb96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce876b1fc4a840588948104f4e7eda5d" is_pre_election: true
I20260809 02:05:19.473560 17010 raft_consensus.cc:2454] T d656709c8cdd43abbb50ca483cec67f2 P 78569db4f20d4cfe884e53818df99efc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 948a4d8653d148859d72dea2948ebb96 in term 0.
I20260809 02:05:19.473672 17155 raft_consensus.cc:2454] T d656709c8cdd43abbb50ca483cec67f2 P ce876b1fc4a840588948104f4e7eda5d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 948a4d8653d148859d72dea2948ebb96 in term 0.
I20260809 02:05:19.474182 17230 leader_election.cc:302] T d656709c8cdd43abbb50ca483cec67f2 P 948a4d8653d148859d72dea2948ebb96 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 78569db4f20d4cfe884e53818df99efc, 948a4d8653d148859d72dea2948ebb96, ce876b1fc4a840588948104f4e7eda5d; no voters:
I20260809 02:05:19.474464 17375 raft_consensus.cc:2781] T d656709c8cdd43abbb50ca483cec67f2 P 948a4d8653d148859d72dea2948ebb96 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:05:19.474545 17375 raft_consensus.cc:482] T d656709c8cdd43abbb50ca483cec67f2 P 948a4d8653d148859d72dea2948ebb96 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:05:19.474575 17375 raft_consensus.cc:3037] T d656709c8cdd43abbb50ca483cec67f2 P 948a4d8653d148859d72dea2948ebb96 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:05:19.475859 17375 raft_consensus.cc:504] T d656709c8cdd43abbb50ca483cec67f2 P 948a4d8653d148859d72dea2948ebb96 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "948a4d8653d148859d72dea2948ebb96" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 38931 } } peers { permanent_uuid: "78569db4f20d4cfe884e53818df99efc" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 44473 } } peers { permanent_uuid: "ce876b1fc4a840588948104f4e7eda5d" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38717 } }
I20260809 02:05:19.476253 17375 leader_election.cc:288] T d656709c8cdd43abbb50ca483cec67f2 P 948a4d8653d148859d72dea2948ebb96 [CANDIDATE]: Term 1 election: Requested vote from peers 78569db4f20d4cfe884e53818df99efc (127.1.192.193:44473), ce876b1fc4a840588948104f4e7eda5d (127.1.192.194:38717)
I20260809 02:05:19.476701 17155 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d656709c8cdd43abbb50ca483cec67f2" candidate_uuid: "948a4d8653d148859d72dea2948ebb96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce876b1fc4a840588948104f4e7eda5d"
I20260809 02:05:19.476706 17010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d656709c8cdd43abbb50ca483cec67f2" candidate_uuid: "948a4d8653d148859d72dea2948ebb96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78569db4f20d4cfe884e53818df99efc"
I20260809 02:05:19.476840 17155 raft_consensus.cc:3037] T d656709c8cdd43abbb50ca483cec67f2 P ce876b1fc4a840588948104f4e7eda5d [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:05:19.476840 17010 raft_consensus.cc:3037] T d656709c8cdd43abbb50ca483cec67f2 P 78569db4f20d4cfe884e53818df99efc [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:05:19.478246 17010 raft_consensus.cc:2454] T d656709c8cdd43abbb50ca483cec67f2 P 78569db4f20d4cfe884e53818df99efc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 948a4d8653d148859d72dea2948ebb96 in term 1.
I20260809 02:05:19.478246 17155 raft_consensus.cc:2454] T d656709c8cdd43abbb50ca483cec67f2 P ce876b1fc4a840588948104f4e7eda5d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 948a4d8653d148859d72dea2948ebb96 in term 1.
I20260809 02:05:19.478559 17231 leader_election.cc:302] T d656709c8cdd43abbb50ca483cec67f2 P 948a4d8653d148859d72dea2948ebb96 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 948a4d8653d148859d72dea2948ebb96, ce876b1fc4a840588948104f4e7eda5d; no voters:
I20260809 02:05:19.478754 17375 raft_consensus.cc:2781] T d656709c8cdd43abbb50ca483cec67f2 P 948a4d8653d148859d72dea2948ebb96 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:05:19.478982 17375 raft_consensus.cc:686] T d656709c8cdd43abbb50ca483cec67f2 P 948a4d8653d148859d72dea2948ebb96 [term 1 LEADER]: Becoming Leader. State: Replica: 948a4d8653d148859d72dea2948ebb96, State: Running, Role: LEADER
I20260809 02:05:19.479337 17375 consensus_queue.cc:237] T d656709c8cdd43abbb50ca483cec67f2 P 948a4d8653d148859d72dea2948ebb96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "948a4d8653d148859d72dea2948ebb96" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 38931 } } peers { permanent_uuid: "78569db4f20d4cfe884e53818df99efc" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 44473 } } peers { permanent_uuid: "ce876b1fc4a840588948104f4e7eda5d" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38717 } }
I20260809 02:05:19.482993 16846 catalog_manager.cc:5495] T d656709c8cdd43abbb50ca483cec67f2 P 948a4d8653d148859d72dea2948ebb96 reported cstate change: term changed from 0 to 1, leader changed from <none> to 948a4d8653d148859d72dea2948ebb96 (127.1.192.195). New cstate: current_term: 1 leader_uuid: "948a4d8653d148859d72dea2948ebb96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "948a4d8653d148859d72dea2948ebb96" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 38931 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "78569db4f20d4cfe884e53818df99efc" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 44473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce876b1fc4a840588948104f4e7eda5d" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38717 } health_report { overall_health: UNKNOWN } } }
I20260809 02:05:19.499701 17365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:19.495748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40572 (local address 127.1.192.195:38931)
0809 02:05:19.495914 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0809 02:05:19.495918 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:05:19.495933 (+ 15us) server_negotiation.cc:407] Connection header received
0809 02:05:19.496008 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:05:19.496011 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:05:19.496067 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:05:19.496166 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:05:19.496434 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:19.497092 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:19.498062 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:19.498304 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:19.498391 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:05:19.498721 (+ 330us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:05:19.499167 (+ 446us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:05:19.499270 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0809 02:05:19.499319 (+ 49us) server_negotiation.cc:299] Negotiation successful
0809 02:05:19.499373 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260809 02:05:19.506269 17155 raft_consensus.cc:1261] T d656709c8cdd43abbb50ca483cec67f2 P ce876b1fc4a840588948104f4e7eda5d [term 1 FOLLOWER]: Refusing update from remote peer 948a4d8653d148859d72dea2948ebb96: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:05:19.506268 17010 raft_consensus.cc:1261] T d656709c8cdd43abbb50ca483cec67f2 P 78569db4f20d4cfe884e53818df99efc [term 1 FOLLOWER]: Refusing update from remote peer 948a4d8653d148859d72dea2948ebb96: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:05:19.507141 17379 consensus_queue.cc:1035] T d656709c8cdd43abbb50ca483cec67f2 P 948a4d8653d148859d72dea2948ebb96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce876b1fc4a840588948104f4e7eda5d" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:05:19.507336 17375 consensus_queue.cc:1035] T d656709c8cdd43abbb50ca483cec67f2 P 948a4d8653d148859d72dea2948ebb96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "78569db4f20d4cfe884e53818df99efc" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 44473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:05:19.515903 17383 mvcc.cc:205] Tried to move back new op lower bound from 7316443625489555456 to 7316443625390731264. Current Snapshot: MvccSnapshot[applied={T|T < 7316443625489555456}]
I20260809 02:05:19.517100 17384 mvcc.cc:205] Tried to move back new op lower bound from 7316443625489555456 to 7316443625390731264. Current Snapshot: MvccSnapshot[applied={T|T < 7316443625489555456}]
I20260809 02:05:19.523473 16841 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54652:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:05:19.523645 16841 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54652:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:05:19.526223 16841 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a63f7b9159694eeebd13d63fa56373e3: Sending DeleteTablet for 3 replicas of tablet d656709c8cdd43abbb50ca483cec67f2
I20260809 02:05:19.527174 17278 tablet_service.cc:1514] Processing DeleteTablet for tablet d656709c8cdd43abbb50ca483cec67f2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:05:19 UTC) from {username='kudu', principal='kudu/127.1.192.254@KRBTEST.COM'} at 127.0.0.1:40564
I20260809 02:05:19.527333 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 16919
I20260809 02:05:19.527300 17134 tablet_service.cc:1514] Processing DeleteTablet for tablet d656709c8cdd43abbb50ca483cec67f2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:05:19 UTC) from {username='kudu', principal='kudu/127.1.192.254@KRBTEST.COM'} at 127.0.0.1:35022
I20260809 02:05:19.529229 17392 tablet_replica.cc:330] T d656709c8cdd43abbb50ca483cec67f2 P ce876b1fc4a840588948104f4e7eda5d: stopping tablet replica
I20260809 02:05:19.529654 17392 raft_consensus.cc:2229] T d656709c8cdd43abbb50ca483cec67f2 P ce876b1fc4a840588948104f4e7eda5d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:05:19.530041 17392 raft_consensus.cc:2258] T d656709c8cdd43abbb50ca483cec67f2 P ce876b1fc4a840588948104f4e7eda5d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:05:19.531764 17392 ts_tablet_manager.cc:1905] T d656709c8cdd43abbb50ca483cec67f2 P ce876b1fc4a840588948104f4e7eda5d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:05:19.534675 17392 ts_tablet_manager.cc:1918] T d656709c8cdd43abbb50ca483cec67f2 P ce876b1fc4a840588948104f4e7eda5d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:05:19.534782 17392 log.cc:1196] T d656709c8cdd43abbb50ca483cec67f2 P ce876b1fc4a840588948104f4e7eda5d: Deleting WAL directory at /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-1/wal/wals/d656709c8cdd43abbb50ca483cec67f2
I20260809 02:05:19.535121 17391 tablet_replica.cc:330] T d656709c8cdd43abbb50ca483cec67f2 P 948a4d8653d148859d72dea2948ebb96: stopping tablet replica
I20260809 02:05:19.535561 17391 raft_consensus.cc:2229] T d656709c8cdd43abbb50ca483cec67f2 P 948a4d8653d148859d72dea2948ebb96 [term 1 LEADER]: Raft consensus shutting down.
I20260809 02:05:19.535610 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 17066
W20260809 02:05:19.535665 16832 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.192.193:44473 (error 108)
I20260809 02:05:19.539104 17391 raft_consensus.cc:2258] T d656709c8cdd43abbb50ca483cec67f2 P 948a4d8653d148859d72dea2948ebb96 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:05:19.541003 17391 ts_tablet_manager.cc:1905] T d656709c8cdd43abbb50ca483cec67f2 P 948a4d8653d148859d72dea2948ebb96: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:05:19.544821 17391 ts_tablet_manager.cc:1918] T d656709c8cdd43abbb50ca483cec67f2 P 948a4d8653d148859d72dea2948ebb96: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:05:19.544955 17391 log.cc:1196] T d656709c8cdd43abbb50ca483cec67f2 P 948a4d8653d148859d72dea2948ebb96: Deleting WAL directory at /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-2/wal/wals/d656709c8cdd43abbb50ca483cec67f2
I20260809 02:05:19.545328 17391 ts_tablet_manager.cc:1939] T d656709c8cdd43abbb50ca483cec67f2 P 948a4d8653d148859d72dea2948ebb96: Deleting consensus metadata
I20260809 02:05:19.546155 17366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:19.545731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.193:44473 (local address 127.0.0.1:58968)
0809 02:05:19.545959 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:19.546051 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.193:44473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137}
W20260809 02:05:19.546448 16832 catalog_manager.cc:4584] TS 78569db4f20d4cfe884e53818df99efc (127.1.192.193:44473): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d656709c8cdd43abbb50ca483cec67f2: Network error: Client connection negotiation failed: client connection to 127.1.192.193:44473: connect: Connection refused (error 111)
I20260809 02:05:19.546840 16829 catalog_manager.cc:4841] TS 948a4d8653d148859d72dea2948ebb96 (127.1.192.195:38931): tablet d656709c8cdd43abbb50ca483cec67f2 (table test-table [id=cc1433a9f55942c79e618388f04df8a0]) successfully deleted
W20260809 02:05:19.546949 17366 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:05:19.546459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.194:38717 (local address 127.0.0.1:35034)
0809 02:05:19.546620 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:19.546635 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 02:05:19.546731 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:05:19.546864 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.194:38717: BlockingWrite error: write error: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":97}
I20260809 02:05:19.547420 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 17211
I20260809 02:05:19.555732 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 16812
I20260809 02:05:19.564095 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.192.254:39909
--webserver_interface=127.1.192.254
--webserver_port=46177
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab
--principal=kudu/127.1.192.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:39143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.192.254:39909
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:05:19.678368 17394 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:05:19.678776 17394 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:05:19.678875 17394 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:05:19.683626 17394 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:05:19.683732 17394 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:05:19.683768 17394 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:05:19.683807 17394 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:05:19.683842 17394 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:05:19.689342 17394 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:39143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.1.192.254:39909
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab
--principal=kudu/127.1.192.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.192.254:39909
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.192.254
--webserver_port=46177
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17394
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:05:19.690660 17394 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:05:19.691740 17394 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:05:19.706497 17400 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:05:19.706478 17403 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:05:19.706714 17394 server_base.cc:1034] running on GCE node
W20260809 02:05:19.706498 17401 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:05:19.707552 17394 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:05:19.708920 17394 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:05:19.710166 17394 hybrid_clock.cc:648] HybridClock initialized: now 1786241119710144 us; error 35 us; skew 500 ppm
Aug 09 02:05:19 dist-test-slave-ss4n krb5kdc[16799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241119, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:05:19.718122 17394 init.cc:377] Logged in from keytab as kudu/127.1.192.254@KRBTEST.COM (short username kudu)
I20260809 02:05:19.719190 17394 webserver.cc:460] Webserver started at http://127.1.192.254:46177/ using document root <none> and password file <none>
I20260809 02:05:19.719779 17394 fs_manager.cc:359] Metadata directory not provided
I20260809 02:05:19.719859 17394 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:05:19.723935 17394 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 02:05:19.726230 17410 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:05:19.727504 17394 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 02:05:19.727644 17394 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/master-0/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/master-0/wal
uuid: "a63f7b9159694eeebd13d63fa56373e3"
format_stamp: "Formatted at 2026-08-09 02:05:18 on dist-test-slave-ss4n"
server_key: "daad3b3cebb0f8a9e1960755b53597af"
server_key_iv: "e5de2a39794e0dffe6a648b932fb4255"
server_key_version: "encryptionkey@0"
I20260809 02:05:19.728296 17394 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:05:19.741402 17394 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:05:19.744683 17394 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:05:19.744902 17394 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:05:19.752924 17394 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.254:39909
I20260809 02:05:19.753322 17465 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.254:39909 every 8 connection(s)
I20260809 02:05:19.754185 17394 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
I20260809 02:05:19.758769 17466 sys_catalog.cc:263] Verifying existing consensus state
I20260809 02:05:19.760540 17466 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a63f7b9159694eeebd13d63fa56373e3: Bootstrap starting.
I20260809 02:05:19.762291 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 17394
I20260809 02:05:19.762799 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.192.193:44473
--local_ip_for_outbound_sockets=127.1.192.193
--tserver_master_addrs=127.1.192.254:39909
--webserver_port=36399
--webserver_interface=127.1.192.193
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.193.keytab
--principal=kudu/127.1.192.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:39143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 02:05:19.779981 17466 log.cc:824] T 00000000000000000000000000000000 P a63f7b9159694eeebd13d63fa56373e3: Log is configured to *not* fsync() on all Append() calls
I20260809 02:05:19.789297 17466 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a63f7b9159694eeebd13d63fa56373e3: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 02:05:19.789767 17466 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a63f7b9159694eeebd13d63fa56373e3: Bootstrap complete.
I20260809 02:05:19.794325 17466 raft_consensus.cc:348] T 00000000000000000000000000000000 P a63f7b9159694eeebd13d63fa56373e3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a63f7b9159694eeebd13d63fa56373e3" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 39909 } }
I20260809 02:05:19.794751 17466 raft_consensus.cc:729] T 00000000000000000000000000000000 P a63f7b9159694eeebd13d63fa56373e3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a63f7b9159694eeebd13d63fa56373e3, State: Initialized, Role: FOLLOWER
I20260809 02:05:19.795243 17466 consensus_queue.cc:260] T 00000000000000000000000000000000 P a63f7b9159694eeebd13d63fa56373e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a63f7b9159694eeebd13d63fa56373e3" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 39909 } }
I20260809 02:05:19.795357 17466 raft_consensus.cc:388] T 00000000000000000000000000000000 P a63f7b9159694eeebd13d63fa56373e3 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:05:19.795450 17466 raft_consensus.cc:482] T 00000000000000000000000000000000 P a63f7b9159694eeebd13d63fa56373e3 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:05:19.795568 17466 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a63f7b9159694eeebd13d63fa56373e3 [term 1 FOLLOWER]: Advancing to term 2
I20260809 02:05:19.797673 17466 raft_consensus.cc:504] T 00000000000000000000000000000000 P a63f7b9159694eeebd13d63fa56373e3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a63f7b9159694eeebd13d63fa56373e3" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 39909 } }
I20260809 02:05:19.798022 17466 leader_election.cc:302] T 00000000000000000000000000000000 P a63f7b9159694eeebd13d63fa56373e3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a63f7b9159694eeebd13d63fa56373e3; no voters:
I20260809 02:05:19.798393 17466 leader_election.cc:288] T 00000000000000000000000000000000 P a63f7b9159694eeebd13d63fa56373e3 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 02:05:19.798499 17471 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a63f7b9159694eeebd13d63fa56373e3 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 02:05:19.798714 17471 raft_consensus.cc:686] T 00000000000000000000000000000000 P a63f7b9159694eeebd13d63fa56373e3 [term 2 LEADER]: Becoming Leader. State: Replica: a63f7b9159694eeebd13d63fa56373e3, State: Running, Role: LEADER
I20260809 02:05:19.799015 17471 consensus_queue.cc:237] T 00000000000000000000000000000000 P a63f7b9159694eeebd13d63fa56373e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a63f7b9159694eeebd13d63fa56373e3" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 39909 } }
I20260809 02:05:19.799719 17466 sys_catalog.cc:564] T 00000000000000000000000000000000 P a63f7b9159694eeebd13d63fa56373e3 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:05:19.800431 17472 sys_catalog.cc:455] T 00000000000000000000000000000000 P a63f7b9159694eeebd13d63fa56373e3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "a63f7b9159694eeebd13d63fa56373e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a63f7b9159694eeebd13d63fa56373e3" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 39909 } } }
I20260809 02:05:19.800573 17472 sys_catalog.cc:458] T 00000000000000000000000000000000 P a63f7b9159694eeebd13d63fa56373e3 [sys.catalog]: This master's current role is: LEADER
I20260809 02:05:19.801223 17479 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:05:19.801270 17473 sys_catalog.cc:455] T 00000000000000000000000000000000 P a63f7b9159694eeebd13d63fa56373e3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a63f7b9159694eeebd13d63fa56373e3. Latest consensus state: current_term: 2 leader_uuid: "a63f7b9159694eeebd13d63fa56373e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a63f7b9159694eeebd13d63fa56373e3" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 39909 } } }
I20260809 02:05:19.801395 17473 sys_catalog.cc:458] T 00000000000000000000000000000000 P a63f7b9159694eeebd13d63fa56373e3 [sys.catalog]: This master's current role is: LEADER
I20260809 02:05:19.806433 17479 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:05:19.807216 17479 catalog_manager.cc:1259] Loaded cluster ID: 83f659d0a92b496f90e277950880f819
I20260809 02:05:19.807337 17479 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:05:19.808972 17479 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260809 02:05:19.813532 17479 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260809 02:05:19.820101 17479 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:05:19.821086 17479 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a63f7b9159694eeebd13d63fa56373e3: Loaded TSK: 0
I20260809 02:05:19.822477 17479 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 02:05:19.889096 17469 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:05:19.889401 17469 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:05:19.889452 17469 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:05:19.895682 17469 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:05:19.895790 17469 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:05:19.895916 17469 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.192.193
I20260809 02:05:19.906315 17469 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:39143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.193.keytab
--principal=kudu/127.1.192.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.192.193:44473
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.192.193
--webserver_port=36399
--tserver_master_addrs=127.1.192.254:39909
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17469
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.192.193
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:05:19.907770 17469 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:05:19.909199 17469 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:05:19.923069 17500 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:05:19.923069 17497 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:05:19.924546 17469 server_base.cc:1034] running on GCE node
W20260809 02:05:19.924635 17498 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:05:19.925194 17469 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:05:19.926146 17469 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:05:19.927454 17469 hybrid_clock.cc:648] HybridClock initialized: now 1786241119927418 us; error 52 us; skew 500 ppm
Aug 09 02:05:19 dist-test-slave-ss4n krb5kdc[16799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241119, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:05:19.931720 17469 init.cc:377] Logged in from keytab as kudu/127.1.192.193@KRBTEST.COM (short username kudu)
I20260809 02:05:19.932909 17469 webserver.cc:460] Webserver started at http://127.1.192.193:36399/ using document root <none> and password file <none>
I20260809 02:05:19.933533 17469 fs_manager.cc:359] Metadata directory not provided
I20260809 02:05:19.933722 17469 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:05:19.939918 17469 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 02:05:19.942492 17508 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:05:19.943959 17469 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 02:05:19.944137 17469 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-0/wal
uuid: "78569db4f20d4cfe884e53818df99efc"
format_stamp: "Formatted at 2026-08-09 02:05:18 on dist-test-slave-ss4n"
server_key: "ffa20ede60e9219240583a07f2b418c8"
server_key_iv: "2afde7e05377b354a77453b2d285e375"
server_key_version: "encryptionkey@0"
I20260809 02:05:19.944769 17469 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:05:19.960886 17469 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:05:19.964586 17469 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:05:19.964814 17469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:05:19.965529 17469 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:05:19.966753 17516 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 02:05:19.971252 17469 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 02:05:19.971331 17469 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20260809 02:05:19.971410 17469 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 02:05:19.973214 17469 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 02:05:19.973277 17469 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260809 02:05:19.973557 17516 tablet_bootstrap.cc:492] T d656709c8cdd43abbb50ca483cec67f2 P 78569db4f20d4cfe884e53818df99efc: Bootstrap starting.
I20260809 02:05:19.984581 17469 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.193:44473
I20260809 02:05:19.984791 17631 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.193:44473 every 8 connection(s)
I20260809 02:05:19.985875 17469 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-0/data/info.pb
I20260809 02:05:19.994777 17516 log.cc:824] T d656709c8cdd43abbb50ca483cec67f2 P 78569db4f20d4cfe884e53818df99efc: Log is configured to *not* fsync() on all Append() calls
I20260809 02:05:19.995105 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 17469
I20260809 02:05:19.995671 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.192.194:38717
--local_ip_for_outbound_sockets=127.1.192.194
--tserver_master_addrs=127.1.192.254:39909
--webserver_port=42009
--webserver_interface=127.1.192.194
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.194.keytab
--principal=kudu/127.1.192.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:39143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 02:05:19 dist-test-slave-ss4n krb5kdc[16799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241119, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.193@KRBTEST.COM for kudu/127.1.192.254@KRBTEST.COM
I20260809 02:05:20.004492 17516 tablet_bootstrap.cc:492] T d656709c8cdd43abbb50ca483cec67f2 P 78569db4f20d4cfe884e53818df99efc: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 02:05:20.005010 17516 tablet_bootstrap.cc:492] T d656709c8cdd43abbb50ca483cec67f2 P 78569db4f20d4cfe884e53818df99efc: Bootstrap complete.
I20260809 02:05:20.006348 17636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:19.987966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:39909 (local address 127.1.192.193:49629)
0809 02:05:19.989068 (+ 1102us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:19.989113 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 02:05:19.990016 (+ 903us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:05:19.991270 (+ 1254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:05:19.991280 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:05:19.991786 (+ 506us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:05:19.992570 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:05:19.992592 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:05:19.994207 (+ 1615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:05:19.994211 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:05:19.995680 (+ 1469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:05:19.995693 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:05:19.997030 (+ 1337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:05:19.997035 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:05:19.997235 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:05:19.997998 (+ 763us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:05:19.998025 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:05:20.000272 (+ 2247us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:05:20.002894 (+ 2622us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:05:20.002900 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:05:20.002914 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:05:20.003202 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:05:20.003582 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:05:20.003586 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:05:20.003588 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:05:20.003732 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:05:20.004494 (+ 762us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:05:20.004503 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:05:20.004789 (+ 286us) client_negotiation.cc:769] Sending connection context
0809 02:05:20.004978 (+ 189us) client_negotiation.cc:240] Negotiation successful
0809 02:05:20.005348 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":906,"thread_start_us":159,"threads_started":1}
I20260809 02:05:20.006335 17635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:19.987996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.193:49629 (local address 127.1.192.254:39909)
0809 02:05:19.988433 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0809 02:05:19.988454 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0809 02:05:19.989252 (+ 798us) server_negotiation.cc:407] Connection header received
0809 02:05:19.990448 (+ 1196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:05:19.990458 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:05:19.990757 (+ 299us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:05:19.991095 (+ 338us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:05:19.992777 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:19.994031 (+ 1254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:19.995854 (+ 1823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:19.996626 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:19.996802 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:05:20.000588 (+ 3786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:05:20.000637 (+ 49us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:05:20.000649 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:05:20.000683 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:05:20.002754 (+ 2071us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:05:20.003341 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:05:20.003354 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:05:20.003358 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:05:20.003479 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:05:20.003834 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:05:20.003838 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:05:20.003840 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:05:20.004298 (+ 458us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:05:20.004433 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0809 02:05:20.005179 (+ 746us) server_negotiation.cc:299] Negotiation successful
0809 02:05:20.005474 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":281,"thread_start_us":149,"threads_started":1}
I20260809 02:05:20.006755 17516 ts_tablet_manager.cc:1397] T d656709c8cdd43abbb50ca483cec67f2 P 78569db4f20d4cfe884e53818df99efc: Time spent bootstrapping tablet: real 0.033s user 0.017s sys 0.014s
I20260809 02:05:20.009174 17632 heartbeater.cc:347] Connected to a master server at 127.1.192.254:39909
I20260809 02:05:20.009441 17632 heartbeater.cc:464] Registering TS with master...
I20260809 02:05:20.009982 17632 heartbeater.cc:511] Master 127.1.192.254:39909 requested a full tablet report, sending...
I20260809 02:05:20.010493 17516 raft_consensus.cc:348] T d656709c8cdd43abbb50ca483cec67f2 P 78569db4f20d4cfe884e53818df99efc [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "948a4d8653d148859d72dea2948ebb96" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 38931 } } peers { permanent_uuid: "78569db4f20d4cfe884e53818df99efc" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 44473 } } peers { permanent_uuid: "ce876b1fc4a840588948104f4e7eda5d" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38717 } }
I20260809 02:05:20.011185 17516 raft_consensus.cc:729] T d656709c8cdd43abbb50ca483cec67f2 P 78569db4f20d4cfe884e53818df99efc [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78569db4f20d4cfe884e53818df99efc, State: Initialized, Role: FOLLOWER
I20260809 02:05:20.011694 17516 consensus_queue.cc:260] T d656709c8cdd43abbb50ca483cec67f2 P 78569db4f20d4cfe884e53818df99efc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "948a4d8653d148859d72dea2948ebb96" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 38931 } } peers { permanent_uuid: "78569db4f20d4cfe884e53818df99efc" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 44473 } } peers { permanent_uuid: "ce876b1fc4a840588948104f4e7eda5d" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38717 } }
I20260809 02:05:20.012624 17427 ts_manager.cc:194] Registered new tserver with Master: 78569db4f20d4cfe884e53818df99efc (127.1.192.193:44473)
I20260809 02:05:20.012737 17516 ts_tablet_manager.cc:1428] T d656709c8cdd43abbb50ca483cec67f2 P 78569db4f20d4cfe884e53818df99efc: Time spent starting tablet: real 0.006s user 0.005s sys 0.001s
I20260809 02:05:20.016465 17427 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.192.193@KRBTEST.COM'} at 127.1.192.193:49629
I20260809 02:05:20.019343 17632 heartbeater.cc:503] Master 127.1.192.254:39909 was elected leader, sending a full tablet report...
Aug 09 02:05:20 dist-test-slave-ss4n krb5kdc[16799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241119, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.254@KRBTEST.COM for kudu/127.1.192.193@KRBTEST.COM
I20260809 02:05:20.028787 17641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:20.015671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58982 (local address 127.1.192.193:44473)
0809 02:05:20.016435 (+ 764us) server_negotiation.cc:206] Beginning negotiation
0809 02:05:20.016446 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 02:05:20.016471 (+ 25us) server_negotiation.cc:407] Connection header received
0809 02:05:20.016543 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:05:20.016548 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:05:20.016756 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:05:20.016894 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:05:20.019570 (+ 2676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:20.020668 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:20.022032 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:20.022413 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:20.022502 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:05:20.025040 (+ 2538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:05:20.025049 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:05:20.025059 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:05:20.025082 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:05:20.026804 (+ 1722us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:05:20.027320 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:05:20.027325 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:05:20.027327 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:05:20.027403 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:05:20.027707 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:05:20.027711 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:05:20.027713 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:05:20.027957 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:05:20.028094 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0809 02:05:20.028445 (+ 351us) server_negotiation.cc:299] Negotiation successful
0809 02:05:20.028619 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":662,"thread_start_us":111,"threads_started":1}
I20260809 02:05:20.028787 17640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:20.015596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.193:44473 (local address 127.0.0.1:58982)
0809 02:05:20.016063 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:20.016115 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 02:05:20.016243 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:05:20.016920 (+ 677us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:05:20.016926 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:05:20.017428 (+ 502us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:05:20.017607 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:05:20.017617 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:05:20.020799 (+ 3182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:05:20.020803 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:05:20.021861 (+ 1058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:05:20.021874 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:05:20.022537 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:05:20.022541 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:05:20.022674 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:05:20.023267 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:05:20.023283 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:05:20.024885 (+ 1602us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:05:20.026929 (+ 2044us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:05:20.026939 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:05:20.026945 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:05:20.027192 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:05:20.027502 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:05:20.027506 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:05:20.027508 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:05:20.027611 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:05:20.028085 (+ 474us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:05:20.028100 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:05:20.028222 (+ 122us) client_negotiation.cc:769] Sending connection context
0809 02:05:20.028362 (+ 140us) client_negotiation.cc:240] Negotiation successful
0809 02:05:20.028594 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":128,"threads_started":1}
I20260809 02:05:20.030169 17561 tablet_service.cc:1514] Processing DeleteTablet for tablet d656709c8cdd43abbb50ca483cec67f2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:05:19 UTC) from {username='kudu', principal='kudu/127.1.192.254@KRBTEST.COM'} at 127.0.0.1:58982
I20260809 02:05:20.031344 17642 tablet_replica.cc:330] stopping tablet replica
I20260809 02:05:20.031738 17642 raft_consensus.cc:2229] T d656709c8cdd43abbb50ca483cec67f2 P 78569db4f20d4cfe884e53818df99efc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:05:20.032027 17642 raft_consensus.cc:2258] T d656709c8cdd43abbb50ca483cec67f2 P 78569db4f20d4cfe884e53818df99efc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:05:20.033221 17642 ts_tablet_manager.cc:1905] T d656709c8cdd43abbb50ca483cec67f2 P 78569db4f20d4cfe884e53818df99efc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:05:20.037248 17642 ts_tablet_manager.cc:1918] T d656709c8cdd43abbb50ca483cec67f2 P 78569db4f20d4cfe884e53818df99efc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:05:20.037351 17642 log.cc:1196] T d656709c8cdd43abbb50ca483cec67f2 P 78569db4f20d4cfe884e53818df99efc: Deleting WAL directory at /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-0/wal/wals/d656709c8cdd43abbb50ca483cec67f2
I20260809 02:05:20.037696 17642 ts_tablet_manager.cc:1939] T d656709c8cdd43abbb50ca483cec67f2 P 78569db4f20d4cfe884e53818df99efc: Deleting consensus metadata
I20260809 02:05:20.040125 17413 catalog_manager.cc:4841] TS 78569db4f20d4cfe884e53818df99efc (127.1.192.193:44473): tablet d656709c8cdd43abbb50ca483cec67f2 (table test-table [id=cc1433a9f55942c79e618388f04df8a0]) successfully deleted
W20260809 02:05:20.122563 17638 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:05:20.123006 17638 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:05:20.123091 17638 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:05:20.127345 17638 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:05:20.127441 17638 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:05:20.127549 17638 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.192.194
I20260809 02:05:20.133574 17638 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:39143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.194.keytab
--principal=kudu/127.1.192.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.192.194:38717
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.192.194
--webserver_port=42009
--tserver_master_addrs=127.1.192.254:39909
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17638
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.192.194
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:05:20.134781 17638 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:05:20.135831 17638 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:05:20.146219 17650 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:05:20.146366 17638 server_base.cc:1034] running on GCE node
W20260809 02:05:20.146219 17647 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:05:20.146238 17648 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:05:20.147087 17638 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:05:20.147915 17638 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:05:20.149147 17638 hybrid_clock.cc:648] HybridClock initialized: now 1786241120149100 us; error 60 us; skew 500 ppm
Aug 09 02:05:20 dist-test-slave-ss4n krb5kdc[16799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241120, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:05:20.152316 17638 init.cc:377] Logged in from keytab as kudu/127.1.192.194@KRBTEST.COM (short username kudu)
I20260809 02:05:20.153219 17638 webserver.cc:460] Webserver started at http://127.1.192.194:42009/ using document root <none> and password file <none>
I20260809 02:05:20.153825 17638 fs_manager.cc:359] Metadata directory not provided
I20260809 02:05:20.153913 17638 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:05:20.157963 17638 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.003s
I20260809 02:05:20.160040 17657 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:05:20.161412 17638 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 02:05:20.161597 17638 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-1/wal
uuid: "ce876b1fc4a840588948104f4e7eda5d"
format_stamp: "Formatted at 2026-08-09 02:05:19 on dist-test-slave-ss4n"
server_key: "62739a957f5623802f673291825320ae"
server_key_iv: "22ee1ca0fb96c18830f71b6ffffcfede"
server_key_version: "encryptionkey@0"
I20260809 02:05:20.162235 17638 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:05:20.182004 17638 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:05:20.185426 17638 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:05:20.185667 17638 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:05:20.186287 17638 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:05:20.187407 17665 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
W20260809 02:05:20.191260 17665 ts_tablet_manager.cc:1852] T d656709c8cdd43abbb50ca483cec67f2 P ce876b1fc4a840588948104f4e7eda5d: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260809 02:05:20.191761 17665 ts_tablet_manager.cc:1905] T d656709c8cdd43abbb50ca483cec67f2 P ce876b1fc4a840588948104f4e7eda5d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:05:20.196029 17665 ts_tablet_manager.cc:1918] T d656709c8cdd43abbb50ca483cec67f2 P ce876b1fc4a840588948104f4e7eda5d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:05:20.196216 17665 ts_tablet_manager.cc:1939] T d656709c8cdd43abbb50ca483cec67f2 P ce876b1fc4a840588948104f4e7eda5d: Deleting consensus metadata
I20260809 02:05:20.196595 17638 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260809 02:05:20.196698 17638 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.010s user 0.000s sys 0.000s
I20260809 02:05:20.196794 17638 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:05:20.196861 17638 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:05:20.207193 17638 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.194:38717
I20260809 02:05:20.207309 17783 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.194:38717 every 8 connection(s)
I20260809 02:05:20.208218 17638 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-1/data/info.pb
I20260809 02:05:20.215698 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 17638
I20260809 02:05:20.216212 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.192.195:38931
--local_ip_for_outbound_sockets=127.1.192.195
--tserver_master_addrs=127.1.192.254:39909
--webserver_port=46035
--webserver_interface=127.1.192.195
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.195.keytab
--principal=kudu/127.1.192.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:39143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 02:05:20 dist-test-slave-ss4n krb5kdc[16799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241120, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.194@KRBTEST.COM for kudu/127.1.192.254@KRBTEST.COM
I20260809 02:05:20.223222 17635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:20.210290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.194:32781 (local address 127.1.192.254:39909)
0809 02:05:20.210446 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0809 02:05:20.210450 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:05:20.211338 (+ 888us) server_negotiation.cc:407] Connection header received
0809 02:05:20.212359 (+ 1021us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:05:20.212363 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:05:20.212431 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:05:20.212587 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:05:20.213879 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:20.214469 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:20.215975 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:20.216250 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:20.216417 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:05:20.219240 (+ 2823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:05:20.219244 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:05:20.219246 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:05:20.219273 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:05:20.220921 (+ 1648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:05:20.221511 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:05:20.221515 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:05:20.221517 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:05:20.221602 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:05:20.221962 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:05:20.221967 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:05:20.221969 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:05:20.222186 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:05:20.222330 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0809 02:05:20.222874 (+ 544us) server_negotiation.cc:299] Negotiation successful
0809 02:05:20.223028 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 02:05:20.224046 17786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:20.210692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:39909 (local address 127.1.192.194:32781)
0809 02:05:20.211170 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:20.211215 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 02:05:20.212135 (+ 920us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:05:20.212765 (+ 630us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:05:20.212772 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:05:20.213145 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:05:20.213718 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:05:20.213735 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:05:20.214631 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:05:20.214634 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:05:20.215780 (+ 1146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:05:20.215794 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:05:20.216542 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:05:20.216552 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:05:20.216711 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:05:20.217382 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:05:20.217399 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:05:20.219032 (+ 1633us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:05:20.221046 (+ 2014us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:05:20.221053 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:05:20.221070 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:05:20.221360 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:05:20.221700 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:05:20.221703 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:05:20.221705 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:05:20.221860 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:05:20.222318 (+ 458us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:05:20.222326 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:05:20.222620 (+ 294us) client_negotiation.cc:769] Sending connection context
0809 02:05:20.222829 (+ 209us) client_negotiation.cc:240] Negotiation successful
0809 02:05:20.223190 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":107,"threads_started":1}
I20260809 02:05:20.225659 17784 heartbeater.cc:347] Connected to a master server at 127.1.192.254:39909
I20260809 02:05:20.225942 17784 heartbeater.cc:464] Registering TS with master...
I20260809 02:05:20.226456 17784 heartbeater.cc:511] Master 127.1.192.254:39909 requested a full tablet report, sending...
I20260809 02:05:20.227676 17427 ts_manager.cc:194] Registered new tserver with Master: ce876b1fc4a840588948104f4e7eda5d (127.1.192.194:38717)
I20260809 02:05:20.228276 17427 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.192.194@KRBTEST.COM'} at 127.1.192.194:32781
W20260809 02:05:20.339623 17787 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:05:20.339988 17787 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:05:20.340070 17787 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:05:20.344519 17787 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:05:20.344623 17787 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:05:20.344717 17787 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.192.195
I20260809 02:05:20.350584 17787 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:39143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.195.keytab
--principal=kudu/127.1.192.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.192.195:38931
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.192.195
--webserver_port=46035
--tserver_master_addrs=127.1.192.254:39909
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17787
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.192.195
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:05:20.351850 17787 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:05:20.352979 17787 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:05:20.363443 17795 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:05:20.363421 17793 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:05:20.363420 17792 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:05:20.363492 17787 server_base.cc:1034] running on GCE node
I20260809 02:05:20.364362 17787 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:05:20.365059 17787 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:05:20.366235 17787 hybrid_clock.cc:648] HybridClock initialized: now 1786241120366216 us; error 31 us; skew 500 ppm
Aug 09 02:05:20 dist-test-slave-ss4n krb5kdc[16799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241120, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:05:20.369642 17787 init.cc:377] Logged in from keytab as kudu/127.1.192.195@KRBTEST.COM (short username kudu)
I20260809 02:05:20.370608 17787 webserver.cc:460] Webserver started at http://127.1.192.195:46035/ using document root <none> and password file <none>
I20260809 02:05:20.371214 17787 fs_manager.cc:359] Metadata directory not provided
I20260809 02:05:20.371335 17787 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:05:20.375533 17787 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 02:05:20.377769 17802 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:05:20.379113 17787 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 02:05:20.379274 17787 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-2/wal
uuid: "948a4d8653d148859d72dea2948ebb96"
format_stamp: "Formatted at 2026-08-09 02:05:19 on dist-test-slave-ss4n"
server_key: "644cd5db26089d8683d1c9b51441edd2"
server_key_iv: "b410d2bded5c3f09667fec9df33a7206"
server_key_version: "encryptionkey@0"
I20260809 02:05:20.379926 17787 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:05:20.408792 17787 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:05:20.412642 17787 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:05:20.412883 17787 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:05:20.413625 17787 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:05:20.414539 17787 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:05:20.414618 17787 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:05:20.414690 17787 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:05:20.414743 17787 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:05:20.424631 17787 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.195:38931
I20260809 02:05:20.424664 17926 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.195:38931 every 8 connection(s)
I20260809 02:05:20.425904 17787 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-2/data/info.pb
I20260809 02:05:20.435284 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 17787
Aug 09 02:05:20 dist-test-slave-ss4n krb5kdc[16799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241120, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.195@KRBTEST.COM for kudu/127.1.192.254@KRBTEST.COM
I20260809 02:05:20.443060 17635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:20.427955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.195:39871 (local address 127.1.192.254:39909)
0809 02:05:20.428121 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0809 02:05:20.428125 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:05:20.429175 (+ 1050us) server_negotiation.cc:407] Connection header received
0809 02:05:20.430325 (+ 1150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:05:20.430328 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:05:20.430376 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:05:20.430454 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:05:20.431834 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:20.432622 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:20.434136 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:20.434386 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:20.434493 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:05:20.438230 (+ 3737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:05:20.438237 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:05:20.438242 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:05:20.438278 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:05:20.440273 (+ 1995us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:05:20.440938 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:05:20.440941 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:05:20.440943 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:05:20.441025 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:05:20.441558 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:05:20.441562 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:05:20.441564 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:05:20.441884 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:05:20.442059 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0809 02:05:20.442710 (+ 651us) server_negotiation.cc:299] Negotiation successful
0809 02:05:20.442856 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 02:05:20.444222 17929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:20.428335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:39909 (local address 127.1.192.195:39871)
0809 02:05:20.429015 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:20.429059 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 02:05:20.430108 (+ 1049us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:05:20.430681 (+ 573us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:05:20.430688 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:05:20.431071 (+ 383us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:05:20.431662 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:05:20.431677 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:05:20.432785 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:05:20.432788 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:05:20.433992 (+ 1204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:05:20.434004 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:05:20.434596 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:05:20.434600 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:05:20.434778 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:05:20.435532 (+ 754us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:05:20.435552 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:05:20.437989 (+ 2437us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:05:20.440422 (+ 2433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:05:20.440434 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:05:20.440450 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:05:20.440772 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:05:20.441128 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:05:20.441131 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:05:20.441133 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:05:20.441358 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:05:20.442028 (+ 670us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:05:20.442036 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:05:20.442364 (+ 328us) client_negotiation.cc:769] Sending connection context
0809 02:05:20.442650 (+ 286us) client_negotiation.cc:240] Negotiation successful
0809 02:05:20.443031 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":113,"threads_started":1}
I20260809 02:05:20.445658 17927 heartbeater.cc:347] Connected to a master server at 127.1.192.254:39909
I20260809 02:05:20.445962 17927 heartbeater.cc:464] Registering TS with master...
I20260809 02:05:20.446612 17927 heartbeater.cc:511] Master 127.1.192.254:39909 requested a full tablet report, sending...
I20260809 02:05:20.447034 17931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:20.436197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54666 (local address 127.1.192.254:39909)
0809 02:05:20.437253 (+ 1056us) server_negotiation.cc:206] Beginning negotiation
0809 02:05:20.437261 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 02:05:20.437280 (+ 19us) server_negotiation.cc:407] Connection header received
0809 02:05:20.437325 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:05:20.437328 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:05:20.437390 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:05:20.437639 (+ 249us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:05:20.438710 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:20.439687 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:20.440874 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:20.441259 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:20.441384 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:05:20.443106 (+ 1722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:05:20.443110 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:05:20.443113 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:05:20.443147 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:05:20.444972 (+ 1825us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:05:20.445529 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:05:20.445533 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:05:20.445535 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:05:20.445633 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:05:20.445965 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:05:20.445967 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:05:20.445969 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:05:20.446216 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:05:20.446377 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0809 02:05:20.446640 (+ 263us) server_negotiation.cc:299] Negotiation successful
0809 02:05:20.446799 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":264,"thread_start_us":105,"threads_started":1}
I20260809 02:05:20.447805 17426 ts_manager.cc:194] Registered new tserver with Master: 948a4d8653d148859d72dea2948ebb96 (127.1.192.195:38931)
I20260809 02:05:20.448498 17426 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.192.195@KRBTEST.COM'} at 127.1.192.195:39871
I20260809 02:05:20.449041 1795 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 02:05:20.454032 17931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:20.449787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54678 (local address 127.1.192.254:39909)
0809 02:05:20.449921 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0809 02:05:20.449926 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:05:20.450133 (+ 207us) server_negotiation.cc:407] Connection header received
0809 02:05:20.450332 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:05:20.450335 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:05:20.450443 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:05:20.450540 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:05:20.450878 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:20.451435 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:20.452378 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:20.452651 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:20.452775 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:05:20.453135 (+ 360us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:05:20.453611 (+ 476us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:05:20.453746 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0809 02:05:20.453815 (+ 69us) server_negotiation.cc:299] Negotiation successful
0809 02:05:20.453876 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 02:05:20.454744 17427 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:54678:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 02:05:20.458158 17427 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:05:20.466574 17561 tablet_service.cc:1467] Processing CreateTablet for tablet 29bdb8f0aa1e47dab98b2812c2da3908 (DEFAULT_TABLE table=test-table [id=3a994d5ea8c44bc393c4c39419385117]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:05:20.467303 17561 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 29bdb8f0aa1e47dab98b2812c2da3908. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:05:20.470259 17516 tablet_bootstrap.cc:492] T 29bdb8f0aa1e47dab98b2812c2da3908 P 78569db4f20d4cfe884e53818df99efc: Bootstrap starting.
I20260809 02:05:20.471184 17516 tablet_bootstrap.cc:654] T 29bdb8f0aa1e47dab98b2812c2da3908 P 78569db4f20d4cfe884e53818df99efc: Neither blocks nor log segments found. Creating new log.
I20260809 02:05:20.472342 17640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:20.465158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.194:38717 (local address 127.0.0.1:35046)
0809 02:05:20.465447 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:20.465463 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 02:05:20.465587 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:05:20.466713 (+ 1126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:05:20.466716 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:05:20.466739 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:05:20.466870 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:05:20.466878 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:05:20.469110 (+ 2232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:05:20.469114 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:05:20.470370 (+ 1256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:05:20.470379 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:05:20.471615 (+ 1236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:05:20.471619 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:05:20.471880 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:05:20.471978 (+ 98us) client_negotiation.cc:769] Sending connection context
0809 02:05:20.472076 (+ 98us) client_negotiation.cc:240] Negotiation successful
0809 02:05:20.472178 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":199,"mutex_wait_us":115,"spinlock_wait_cycles":1408}
I20260809 02:05:20.472402 17936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:20.465432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35046 (local address 127.1.192.194:38717)
0809 02:05:20.466137 (+ 705us) server_negotiation.cc:206] Beginning negotiation
0809 02:05:20.466151 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 02:05:20.466176 (+ 25us) server_negotiation.cc:407] Connection header received
0809 02:05:20.466302 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:05:20.466306 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:05:20.466536 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:05:20.466811 (+ 275us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:05:20.467056 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:20.468917 (+ 1861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:20.470528 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:20.471442 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:20.471859 (+ 417us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:05:20.472020 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0809 02:05:20.472121 (+ 101us) server_negotiation.cc:299] Negotiation successful
0809 02:05:20.472217 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":584,"thread_start_us":267,"threads_started":1}
I20260809 02:05:20.472930 17938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:20.465776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40578 (local address 127.1.192.195:38931)
0809 02:05:20.466133 (+ 357us) server_negotiation.cc:206] Beginning negotiation
0809 02:05:20.466147 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 02:05:20.466173 (+ 26us) server_negotiation.cc:407] Connection header received
0809 02:05:20.466255 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:05:20.466261 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:05:20.466496 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:05:20.466670 (+ 174us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:05:20.467284 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:20.468524 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:20.470485 (+ 1961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:20.471442 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:20.471583 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:05:20.471763 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0809 02:05:20.472715 (+ 952us) server_negotiation.cc:299] Negotiation successful
0809 02:05:20.472819 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":223,"thread_start_us":137,"threads_started":1}
I20260809 02:05:20.472339 17937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:20.465692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.195:38931 (local address 127.0.0.1:40578)
0809 02:05:20.466026 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:20.466043 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 02:05:20.466138 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:05:20.466671 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:05:20.466675 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:05:20.466701 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:05:20.467139 (+ 438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:05:20.467150 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:05:20.468667 (+ 1517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:05:20.468673 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:05:20.470184 (+ 1511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:05:20.470197 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:05:20.471614 (+ 1417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:05:20.471620 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:05:20.471877 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:05:20.471962 (+ 85us) client_negotiation.cc:769] Sending connection context
0809 02:05:20.472058 (+ 96us) client_negotiation.cc:240] Negotiation successful
0809 02:05:20.472165 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"mutex_wait_us":93,"thread_start_us":114,"threads_started":1}
I20260809 02:05:20.474066 17516 tablet_bootstrap.cc:492] T 29bdb8f0aa1e47dab98b2812c2da3908 P 78569db4f20d4cfe884e53818df99efc: No bootstrap required, opened a new log
I20260809 02:05:20.474167 17516 ts_tablet_manager.cc:1397] T 29bdb8f0aa1e47dab98b2812c2da3908 P 78569db4f20d4cfe884e53818df99efc: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s
I20260809 02:05:20.474133 17711 tablet_service.cc:1467] Processing CreateTablet for tablet 29bdb8f0aa1e47dab98b2812c2da3908 (DEFAULT_TABLE table=test-table [id=3a994d5ea8c44bc393c4c39419385117]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:05:20.474632 17516 raft_consensus.cc:348] T 29bdb8f0aa1e47dab98b2812c2da3908 P 78569db4f20d4cfe884e53818df99efc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce876b1fc4a840588948104f4e7eda5d" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38717 } } peers { permanent_uuid: "948a4d8653d148859d72dea2948ebb96" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 38931 } } peers { permanent_uuid: "78569db4f20d4cfe884e53818df99efc" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 44473 } }
I20260809 02:05:20.474751 17516 raft_consensus.cc:374] T 29bdb8f0aa1e47dab98b2812c2da3908 P 78569db4f20d4cfe884e53818df99efc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:05:20.474788 17516 raft_consensus.cc:729] T 29bdb8f0aa1e47dab98b2812c2da3908 P 78569db4f20d4cfe884e53818df99efc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78569db4f20d4cfe884e53818df99efc, State: Initialized, Role: FOLLOWER
I20260809 02:05:20.474920 17516 consensus_queue.cc:260] T 29bdb8f0aa1e47dab98b2812c2da3908 P 78569db4f20d4cfe884e53818df99efc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce876b1fc4a840588948104f4e7eda5d" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38717 } } peers { permanent_uuid: "948a4d8653d148859d72dea2948ebb96" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 38931 } } peers { permanent_uuid: "78569db4f20d4cfe884e53818df99efc" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 44473 } }
I20260809 02:05:20.475203 17516 ts_tablet_manager.cc:1428] T 29bdb8f0aa1e47dab98b2812c2da3908 P 78569db4f20d4cfe884e53818df99efc: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260809 02:05:20.475309 17711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 29bdb8f0aa1e47dab98b2812c2da3908. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:05:20.476274 17853 tablet_service.cc:1467] Processing CreateTablet for tablet 29bdb8f0aa1e47dab98b2812c2da3908 (DEFAULT_TABLE table=test-table [id=3a994d5ea8c44bc393c4c39419385117]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:05:20.477346 17853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 29bdb8f0aa1e47dab98b2812c2da3908. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:05:20.479961 17665 tablet_bootstrap.cc:492] T 29bdb8f0aa1e47dab98b2812c2da3908 P ce876b1fc4a840588948104f4e7eda5d: Bootstrap starting.
I20260809 02:05:20.482815 17665 tablet_bootstrap.cc:654] T 29bdb8f0aa1e47dab98b2812c2da3908 P ce876b1fc4a840588948104f4e7eda5d: Neither blocks nor log segments found. Creating new log.
I20260809 02:05:20.483850 17665 log.cc:824] T 29bdb8f0aa1e47dab98b2812c2da3908 P ce876b1fc4a840588948104f4e7eda5d: Log is configured to *not* fsync() on all Append() calls
I20260809 02:05:20.486286 17665 tablet_bootstrap.cc:492] T 29bdb8f0aa1e47dab98b2812c2da3908 P ce876b1fc4a840588948104f4e7eda5d: No bootstrap required, opened a new log
I20260809 02:05:20.486487 17665 ts_tablet_manager.cc:1397] T 29bdb8f0aa1e47dab98b2812c2da3908 P ce876b1fc4a840588948104f4e7eda5d: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260809 02:05:20.486462 17941 tablet_bootstrap.cc:492] T 29bdb8f0aa1e47dab98b2812c2da3908 P 948a4d8653d148859d72dea2948ebb96: Bootstrap starting.
W20260809 02:05:20.487110 17634 tablet.cc:2369] T 29bdb8f0aa1e47dab98b2812c2da3908 P 78569db4f20d4cfe884e53818df99efc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 02:05:20.488960 17941 tablet_bootstrap.cc:654] T 29bdb8f0aa1e47dab98b2812c2da3908 P 948a4d8653d148859d72dea2948ebb96: Neither blocks nor log segments found. Creating new log.
I20260809 02:05:20.489830 17941 log.cc:824] T 29bdb8f0aa1e47dab98b2812c2da3908 P 948a4d8653d148859d72dea2948ebb96: Log is configured to *not* fsync() on all Append() calls
I20260809 02:05:20.489936 17665 raft_consensus.cc:348] T 29bdb8f0aa1e47dab98b2812c2da3908 P ce876b1fc4a840588948104f4e7eda5d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce876b1fc4a840588948104f4e7eda5d" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38717 } } peers { permanent_uuid: "948a4d8653d148859d72dea2948ebb96" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 38931 } } peers { permanent_uuid: "78569db4f20d4cfe884e53818df99efc" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 44473 } }
I20260809 02:05:20.490257 17665 raft_consensus.cc:374] T 29bdb8f0aa1e47dab98b2812c2da3908 P ce876b1fc4a840588948104f4e7eda5d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:05:20.490340 17665 raft_consensus.cc:729] T 29bdb8f0aa1e47dab98b2812c2da3908 P ce876b1fc4a840588948104f4e7eda5d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce876b1fc4a840588948104f4e7eda5d, State: Initialized, Role: FOLLOWER
I20260809 02:05:20.490847 17665 consensus_queue.cc:260] T 29bdb8f0aa1e47dab98b2812c2da3908 P ce876b1fc4a840588948104f4e7eda5d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce876b1fc4a840588948104f4e7eda5d" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38717 } } peers { permanent_uuid: "948a4d8653d148859d72dea2948ebb96" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 38931 } } peers { permanent_uuid: "78569db4f20d4cfe884e53818df99efc" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 44473 } }
I20260809 02:05:20.491680 17784 heartbeater.cc:503] Master 127.1.192.254:39909 was elected leader, sending a full tablet report...
I20260809 02:05:20.491917 17665 ts_tablet_manager.cc:1428] T 29bdb8f0aa1e47dab98b2812c2da3908 P ce876b1fc4a840588948104f4e7eda5d: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260809 02:05:20.492308 17941 tablet_bootstrap.cc:492] T 29bdb8f0aa1e47dab98b2812c2da3908 P 948a4d8653d148859d72dea2948ebb96: No bootstrap required, opened a new log
I20260809 02:05:20.492530 17941 ts_tablet_manager.cc:1397] T 29bdb8f0aa1e47dab98b2812c2da3908 P 948a4d8653d148859d72dea2948ebb96: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260809 02:05:20.496588 17941 raft_consensus.cc:348] T 29bdb8f0aa1e47dab98b2812c2da3908 P 948a4d8653d148859d72dea2948ebb96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce876b1fc4a840588948104f4e7eda5d" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38717 } } peers { permanent_uuid: "948a4d8653d148859d72dea2948ebb96" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 38931 } } peers { permanent_uuid: "78569db4f20d4cfe884e53818df99efc" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 44473 } }
I20260809 02:05:20.496847 17941 raft_consensus.cc:374] T 29bdb8f0aa1e47dab98b2812c2da3908 P 948a4d8653d148859d72dea2948ebb96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:05:20.496884 17941 raft_consensus.cc:729] T 29bdb8f0aa1e47dab98b2812c2da3908 P 948a4d8653d148859d72dea2948ebb96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 948a4d8653d148859d72dea2948ebb96, State: Initialized, Role: FOLLOWER
I20260809 02:05:20.497313 17941 consensus_queue.cc:260] T 29bdb8f0aa1e47dab98b2812c2da3908 P 948a4d8653d148859d72dea2948ebb96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce876b1fc4a840588948104f4e7eda5d" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38717 } } peers { permanent_uuid: "948a4d8653d148859d72dea2948ebb96" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 38931 } } peers { permanent_uuid: "78569db4f20d4cfe884e53818df99efc" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 44473 } }
I20260809 02:05:20.498155 17927 heartbeater.cc:503] Master 127.1.192.254:39909 was elected leader, sending a full tablet report...
I20260809 02:05:20.498308 17941 ts_tablet_manager.cc:1428] T 29bdb8f0aa1e47dab98b2812c2da3908 P 948a4d8653d148859d72dea2948ebb96: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260809 02:05:20.551430 17944 raft_consensus.cc:482] T 29bdb8f0aa1e47dab98b2812c2da3908 P 948a4d8653d148859d72dea2948ebb96 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:05:20.551733 17944 raft_consensus.cc:504] T 29bdb8f0aa1e47dab98b2812c2da3908 P 948a4d8653d148859d72dea2948ebb96 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce876b1fc4a840588948104f4e7eda5d" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38717 } } peers { permanent_uuid: "948a4d8653d148859d72dea2948ebb96" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 38931 } } peers { permanent_uuid: "78569db4f20d4cfe884e53818df99efc" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 44473 } }
I20260809 02:05:20.553169 17944 leader_election.cc:288] T 29bdb8f0aa1e47dab98b2812c2da3908 P 948a4d8653d148859d72dea2948ebb96 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ce876b1fc4a840588948104f4e7eda5d (127.1.192.194:38717), 78569db4f20d4cfe884e53818df99efc (127.1.192.193:44473)
I20260809 02:05:20.559533 17929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:20.553435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.193:44473 (local address 127.1.192.195:57867)
0809 02:05:20.553566 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:20.553581 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 02:05:20.553687 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:05:20.554428 (+ 741us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:05:20.554432 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:05:20.554479 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:05:20.554705 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:05:20.554714 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:05:20.555980 (+ 1266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:05:20.555984 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:05:20.557919 (+ 1935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:05:20.557931 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:05:20.559060 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:05:20.559066 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:05:20.559196 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:05:20.559223 (+ 27us) client_negotiation.cc:769] Sending connection context
0809 02:05:20.559308 (+ 85us) client_negotiation.cc:240] Negotiation successful
0809 02:05:20.559386 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32,"mutex_wait_us":407,"spinlock_wait_cycles":9472}
I20260809 02:05:20.559670 17947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:20.553732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.195:57867 (local address 127.1.192.193:44473)
0809 02:05:20.554081 (+ 349us) server_negotiation.cc:206] Beginning negotiation
0809 02:05:20.554088 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 02:05:20.554107 (+ 19us) server_negotiation.cc:407] Connection header received
0809 02:05:20.554166 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:05:20.554169 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:05:20.554242 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:05:20.554378 (+ 136us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:05:20.554881 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:20.555854 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:20.558126 (+ 2272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:20.558917 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:20.559037 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:05:20.559194 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0809 02:05:20.559368 (+ 174us) server_negotiation.cc:299] Negotiation successful
0809 02:05:20.559451 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":235,"thread_start_us":121,"threads_started":1}
I20260809 02:05:20.560102 17946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:20.553435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.194:38717 (local address 127.1.192.195:53771)
0809 02:05:20.553953 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:20.553965 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:05:20.554047 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:05:20.554428 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:05:20.554432 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:05:20.554477 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:05:20.554706 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:05:20.554714 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:05:20.555980 (+ 1266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:05:20.555984 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:05:20.557919 (+ 1935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:05:20.557931 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:05:20.558944 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:05:20.558947 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:05:20.559702 (+ 755us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:05:20.559729 (+ 27us) client_negotiation.cc:769] Sending connection context
0809 02:05:20.559862 (+ 133us) client_negotiation.cc:240] Negotiation successful
0809 02:05:20.559937 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":446,"mutex_wait_us":395,"spinlock_wait_cycles":30336,"thread_start_us":152,"threads_started":1}
I20260809 02:05:20.560129 17936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:20.553614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.195:53771 (local address 127.1.192.194:38717)
0809 02:05:20.553781 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0809 02:05:20.553786 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:05:20.554064 (+ 278us) server_negotiation.cc:407] Connection header received
0809 02:05:20.554149 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:05:20.554152 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:05:20.554220 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:05:20.554349 (+ 129us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:05:20.554882 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:20.555795 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:20.558124 (+ 2329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:20.558792 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:20.558927 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:05:20.559023 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0809 02:05:20.559893 (+ 870us) server_negotiation.cc:299] Negotiation successful
0809 02:05:20.559987 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260809 02:05:20.560326 17584 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29bdb8f0aa1e47dab98b2812c2da3908" candidate_uuid: "948a4d8653d148859d72dea2948ebb96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78569db4f20d4cfe884e53818df99efc" is_pre_election: true
I20260809 02:05:20.560698 17584 raft_consensus.cc:2454] T 29bdb8f0aa1e47dab98b2812c2da3908 P 78569db4f20d4cfe884e53818df99efc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 948a4d8653d148859d72dea2948ebb96 in term 0.
I20260809 02:05:20.560758 17733 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29bdb8f0aa1e47dab98b2812c2da3908" candidate_uuid: "948a4d8653d148859d72dea2948ebb96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce876b1fc4a840588948104f4e7eda5d" is_pre_election: true
I20260809 02:05:20.561100 17733 raft_consensus.cc:2454] T 29bdb8f0aa1e47dab98b2812c2da3908 P ce876b1fc4a840588948104f4e7eda5d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 948a4d8653d148859d72dea2948ebb96 in term 0.
I20260809 02:05:20.561296 17805 leader_election.cc:302] T 29bdb8f0aa1e47dab98b2812c2da3908 P 948a4d8653d148859d72dea2948ebb96 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 78569db4f20d4cfe884e53818df99efc, 948a4d8653d148859d72dea2948ebb96; no voters:
I20260809 02:05:20.561661 17944 raft_consensus.cc:2781] T 29bdb8f0aa1e47dab98b2812c2da3908 P 948a4d8653d148859d72dea2948ebb96 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:05:20.561753 17944 raft_consensus.cc:482] T 29bdb8f0aa1e47dab98b2812c2da3908 P 948a4d8653d148859d72dea2948ebb96 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:05:20.561798 17944 raft_consensus.cc:3037] T 29bdb8f0aa1e47dab98b2812c2da3908 P 948a4d8653d148859d72dea2948ebb96 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:05:20.562915 17944 raft_consensus.cc:504] T 29bdb8f0aa1e47dab98b2812c2da3908 P 948a4d8653d148859d72dea2948ebb96 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce876b1fc4a840588948104f4e7eda5d" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38717 } } peers { permanent_uuid: "948a4d8653d148859d72dea2948ebb96" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 38931 } } peers { permanent_uuid: "78569db4f20d4cfe884e53818df99efc" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 44473 } }
I20260809 02:05:20.563270 17944 leader_election.cc:288] T 29bdb8f0aa1e47dab98b2812c2da3908 P 948a4d8653d148859d72dea2948ebb96 [CANDIDATE]: Term 1 election: Requested vote from peers ce876b1fc4a840588948104f4e7eda5d (127.1.192.194:38717), 78569db4f20d4cfe884e53818df99efc (127.1.192.193:44473)
I20260809 02:05:20.563627 17733 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29bdb8f0aa1e47dab98b2812c2da3908" candidate_uuid: "948a4d8653d148859d72dea2948ebb96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce876b1fc4a840588948104f4e7eda5d"
I20260809 02:05:20.563625 17584 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29bdb8f0aa1e47dab98b2812c2da3908" candidate_uuid: "948a4d8653d148859d72dea2948ebb96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78569db4f20d4cfe884e53818df99efc"
I20260809 02:05:20.563791 17584 raft_consensus.cc:3037] T 29bdb8f0aa1e47dab98b2812c2da3908 P 78569db4f20d4cfe884e53818df99efc [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:05:20.563786 17733 raft_consensus.cc:3037] T 29bdb8f0aa1e47dab98b2812c2da3908 P ce876b1fc4a840588948104f4e7eda5d [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:05:20.565114 17584 raft_consensus.cc:2454] T 29bdb8f0aa1e47dab98b2812c2da3908 P 78569db4f20d4cfe884e53818df99efc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 948a4d8653d148859d72dea2948ebb96 in term 1.
I20260809 02:05:20.565114 17733 raft_consensus.cc:2454] T 29bdb8f0aa1e47dab98b2812c2da3908 P ce876b1fc4a840588948104f4e7eda5d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 948a4d8653d148859d72dea2948ebb96 in term 1.
I20260809 02:05:20.565430 17806 leader_election.cc:302] T 29bdb8f0aa1e47dab98b2812c2da3908 P 948a4d8653d148859d72dea2948ebb96 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 948a4d8653d148859d72dea2948ebb96, ce876b1fc4a840588948104f4e7eda5d; no voters:
I20260809 02:05:20.565631 17944 raft_consensus.cc:2781] T 29bdb8f0aa1e47dab98b2812c2da3908 P 948a4d8653d148859d72dea2948ebb96 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:05:20.565886 17944 raft_consensus.cc:686] T 29bdb8f0aa1e47dab98b2812c2da3908 P 948a4d8653d148859d72dea2948ebb96 [term 1 LEADER]: Becoming Leader. State: Replica: 948a4d8653d148859d72dea2948ebb96, State: Running, Role: LEADER
I20260809 02:05:20.566283 17944 consensus_queue.cc:237] T 29bdb8f0aa1e47dab98b2812c2da3908 P 948a4d8653d148859d72dea2948ebb96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce876b1fc4a840588948104f4e7eda5d" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38717 } } peers { permanent_uuid: "948a4d8653d148859d72dea2948ebb96" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 38931 } } peers { permanent_uuid: "78569db4f20d4cfe884e53818df99efc" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 44473 } }
I20260809 02:05:20.569718 17426 catalog_manager.cc:5495] T 29bdb8f0aa1e47dab98b2812c2da3908 P 948a4d8653d148859d72dea2948ebb96 reported cstate change: term changed from 0 to 1, leader changed from <none> to 948a4d8653d148859d72dea2948ebb96 (127.1.192.195). New cstate: current_term: 1 leader_uuid: "948a4d8653d148859d72dea2948ebb96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce876b1fc4a840588948104f4e7eda5d" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38717 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "948a4d8653d148859d72dea2948ebb96" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 38931 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "78569db4f20d4cfe884e53818df99efc" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 44473 } health_report { overall_health: UNKNOWN } } }
I20260809 02:05:20.600857 17938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:20.597078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40592 (local address 127.1.192.195:38931)
0809 02:05:20.597230 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0809 02:05:20.597234 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:05:20.597250 (+ 16us) server_negotiation.cc:407] Connection header received
0809 02:05:20.597398 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:05:20.597401 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:05:20.597455 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:05:20.597693 (+ 238us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:05:20.597942 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:20.598456 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:20.599327 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:20.599568 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:20.599690 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:05:20.600010 (+ 320us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:05:20.600451 (+ 441us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:05:20.600612 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0809 02:05:20.600650 (+ 38us) server_negotiation.cc:299] Negotiation successful
0809 02:05:20.600708 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 02:05:20.607555 17584 raft_consensus.cc:1261] T 29bdb8f0aa1e47dab98b2812c2da3908 P 78569db4f20d4cfe884e53818df99efc [term 1 FOLLOWER]: Refusing update from remote peer 948a4d8653d148859d72dea2948ebb96: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:05:20.607591 17733 raft_consensus.cc:1261] T 29bdb8f0aa1e47dab98b2812c2da3908 P ce876b1fc4a840588948104f4e7eda5d [term 1 FOLLOWER]: Refusing update from remote peer 948a4d8653d148859d72dea2948ebb96: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:05:20.608546 17948 consensus_queue.cc:1035] T 29bdb8f0aa1e47dab98b2812c2da3908 P 948a4d8653d148859d72dea2948ebb96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "78569db4f20d4cfe884e53818df99efc" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 44473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:05:20.608809 17944 consensus_queue.cc:1035] T 29bdb8f0aa1e47dab98b2812c2da3908 P 948a4d8653d148859d72dea2948ebb96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce876b1fc4a840588948104f4e7eda5d" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:05:20.616803 17953 mvcc.cc:205] Tried to move back new op lower bound from 7316443629999292416 to 7316443629842198528. Current Snapshot: MvccSnapshot[applied={T|T < 7316443629999292416}]
I20260809 02:05:20.617942 17954 mvcc.cc:205] Tried to move back new op lower bound from 7316443629999292416 to 7316443629842198528. Current Snapshot: MvccSnapshot[applied={T|T < 7316443629999292416}]
I20260809 02:05:20.625429 17426 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:54678:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:05:20.625691 17426 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:54678:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:05:20.628435 17426 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a63f7b9159694eeebd13d63fa56373e3: Sending DeleteTablet for 3 replicas of tablet 29bdb8f0aa1e47dab98b2812c2da3908
I20260809 02:05:20.629280 17711 tablet_service.cc:1514] Processing DeleteTablet for tablet 29bdb8f0aa1e47dab98b2812c2da3908 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:05:20 UTC) from {username='kudu', principal='kudu/127.1.192.254@KRBTEST.COM'} at 127.0.0.1:35046
I20260809 02:05:20.629735 17853 tablet_service.cc:1514] Processing DeleteTablet for tablet 29bdb8f0aa1e47dab98b2812c2da3908 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:05:20 UTC) from {username='kudu', principal='kudu/127.1.192.254@KRBTEST.COM'} at 127.0.0.1:40578
I20260809 02:05:20.629880 17561 tablet_service.cc:1514] Processing DeleteTablet for tablet 29bdb8f0aa1e47dab98b2812c2da3908 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:05:20 UTC) from {username='kudu', principal='kudu/127.1.192.254@KRBTEST.COM'} at 127.0.0.1:58982
I20260809 02:05:20.630172 17962 tablet_replica.cc:330] T 29bdb8f0aa1e47dab98b2812c2da3908 P ce876b1fc4a840588948104f4e7eda5d: stopping tablet replica
I20260809 02:05:20.630273 17964 tablet_replica.cc:330] T 29bdb8f0aa1e47dab98b2812c2da3908 P 78569db4f20d4cfe884e53818df99efc: stopping tablet replica
I20260809 02:05:20.630381 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 17469
I20260809 02:05:20.630415 17964 raft_consensus.cc:2229] T 29bdb8f0aa1e47dab98b2812c2da3908 P 78569db4f20d4cfe884e53818df99efc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:05:20.632134 17962 raft_consensus.cc:2229] T 29bdb8f0aa1e47dab98b2812c2da3908 P ce876b1fc4a840588948104f4e7eda5d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:05:20.632543 17962 raft_consensus.cc:2258] T 29bdb8f0aa1e47dab98b2812c2da3908 P ce876b1fc4a840588948104f4e7eda5d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:05:20.634456 17962 ts_tablet_manager.cc:1905] T 29bdb8f0aa1e47dab98b2812c2da3908 P ce876b1fc4a840588948104f4e7eda5d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:05:20.635139 17963 tablet_replica.cc:330] T 29bdb8f0aa1e47dab98b2812c2da3908 P 948a4d8653d148859d72dea2948ebb96: stopping tablet replica
I20260809 02:05:20.635524 17963 raft_consensus.cc:2229] T 29bdb8f0aa1e47dab98b2812c2da3908 P 948a4d8653d148859d72dea2948ebb96 [term 1 LEADER]: Raft consensus shutting down.
I20260809 02:05:20.636018 17963 raft_consensus.cc:2258] T 29bdb8f0aa1e47dab98b2812c2da3908 P 948a4d8653d148859d72dea2948ebb96 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:05:20.637627 17962 ts_tablet_manager.cc:1918] T 29bdb8f0aa1e47dab98b2812c2da3908 P ce876b1fc4a840588948104f4e7eda5d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:05:20.637741 17962 log.cc:1196] T 29bdb8f0aa1e47dab98b2812c2da3908 P ce876b1fc4a840588948104f4e7eda5d: Deleting WAL directory at /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-1/wal/wals/29bdb8f0aa1e47dab98b2812c2da3908
I20260809 02:05:20.638006 17963 ts_tablet_manager.cc:1905] T 29bdb8f0aa1e47dab98b2812c2da3908 P 948a4d8653d148859d72dea2948ebb96: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:05:20.638123 17962 ts_tablet_manager.cc:1939] T 29bdb8f0aa1e47dab98b2812c2da3908 P ce876b1fc4a840588948104f4e7eda5d: Deleting consensus metadata
I20260809 02:05:20.639617 17414 catalog_manager.cc:4841] TS ce876b1fc4a840588948104f4e7eda5d (127.1.192.194:38717): tablet 29bdb8f0aa1e47dab98b2812c2da3908 (table test-table [id=3a994d5ea8c44bc393c4c39419385117]) successfully deleted
I20260809 02:05:20.640935 17963 ts_tablet_manager.cc:1918] T 29bdb8f0aa1e47dab98b2812c2da3908 P 948a4d8653d148859d72dea2948ebb96: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:05:20.641068 17963 log.cc:1196] T 29bdb8f0aa1e47dab98b2812c2da3908 P 948a4d8653d148859d72dea2948ebb96: Deleting WAL directory at /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-2/wal/wals/29bdb8f0aa1e47dab98b2812c2da3908
I20260809 02:05:20.641407 17963 ts_tablet_manager.cc:1939] T 29bdb8f0aa1e47dab98b2812c2da3908 P 948a4d8653d148859d72dea2948ebb96: Deleting consensus metadata
W20260809 02:05:20.641860 17413 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.192.193:44473 (error 108)
I20260809 02:05:20.642589 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 17638
I20260809 02:05:20.643100 17937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:20.642725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.193:44473 (local address 127.0.0.1:58992)
0809 02:05:20.642859 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:20.642981 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.193:44473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260809 02:05:20.643287 17413 catalog_manager.cc:4584] TS 78569db4f20d4cfe884e53818df99efc (127.1.192.193:44473): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 29bdb8f0aa1e47dab98b2812c2da3908: Network error: Client connection negotiation failed: client connection to 127.1.192.193:44473: connect: Connection refused (error 111)
I20260809 02:05:20.644763 17411 catalog_manager.cc:4841] TS 948a4d8653d148859d72dea2948ebb96 (127.1.192.195:38931): tablet 29bdb8f0aa1e47dab98b2812c2da3908 (table test-table [id=3a994d5ea8c44bc393c4c39419385117]) successfully deleted
I20260809 02:05:20.651404 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 17787
I20260809 02:05:20.659269 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 17394
I20260809 02:05:20.668421 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.192.254:39909
--webserver_interface=127.1.192.254
--webserver_port=46177
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab
--principal=kudu/127.1.192.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:39143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.192.254:39909
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:05:20.785708 17966 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:05:20.785979 17966 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:05:20.786020 17966 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:05:20.790258 17966 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:05:20.790329 17966 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:05:20.790349 17966 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:05:20.790366 17966 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:05:20.790381 17966 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:05:20.796162 17966 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:39143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.192.254:39909
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab
--principal=kudu/127.1.192.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.192.254:39909
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.192.254
--webserver_port=46177
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17966
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:05:20.797220 17966 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:05:20.798233 17966 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:05:20.807113 17975 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:05:20.807113 17972 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:05:20.807121 17973 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:05:20.807742 17966 server_base.cc:1034] running on GCE node
I20260809 02:05:20.808374 17966 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:05:20.809722 17966 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:05:20.810926 17966 hybrid_clock.cc:648] HybridClock initialized: now 1786241120810901 us; error 44 us; skew 500 ppm
Aug 09 02:05:20 dist-test-slave-ss4n krb5kdc[16799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241120, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:05:20.820101 17966 init.cc:377] Logged in from keytab as kudu/127.1.192.254@KRBTEST.COM (short username kudu)
I20260809 02:05:20.821144 17966 webserver.cc:460] Webserver started at http://127.1.192.254:46177/ using document root <none> and password file <none>
I20260809 02:05:20.821817 17966 fs_manager.cc:359] Metadata directory not provided
I20260809 02:05:20.821892 17966 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:05:20.826139 17966 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260809 02:05:20.828445 17983 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:05:20.829957 17966 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 02:05:20.830075 17966 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/master-0/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/master-0/wal
uuid: "a63f7b9159694eeebd13d63fa56373e3"
format_stamp: "Formatted at 2026-08-09 02:05:18 on dist-test-slave-ss4n"
server_key: "daad3b3cebb0f8a9e1960755b53597af"
server_key_iv: "e5de2a39794e0dffe6a648b932fb4255"
server_key_version: "encryptionkey@0"
I20260809 02:05:20.830647 17966 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:05:20.859916 17966 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:05:20.863679 17966 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:05:20.863860 17966 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:05:20.872406 17966 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.254:39909
I20260809 02:05:20.872740 18038 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.254:39909 every 8 connection(s)
I20260809 02:05:20.873819 17966 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
I20260809 02:05:20.876667 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 17966
I20260809 02:05:20.877199 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.192.193:44473
--local_ip_for_outbound_sockets=127.1.192.193
--tserver_master_addrs=127.1.192.254:39909
--webserver_port=36399
--webserver_interface=127.1.192.193
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.193.keytab
--principal=kudu/127.1.192.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:39143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 02:05:20.878539 18039 sys_catalog.cc:263] Verifying existing consensus state
I20260809 02:05:20.880276 18039 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a63f7b9159694eeebd13d63fa56373e3: Bootstrap starting.
I20260809 02:05:20.906273 18039 log.cc:824] T 00000000000000000000000000000000 P a63f7b9159694eeebd13d63fa56373e3: Log is configured to *not* fsync() on all Append() calls
I20260809 02:05:20.920975 18039 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a63f7b9159694eeebd13d63fa56373e3: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 02:05:20.921571 18039 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a63f7b9159694eeebd13d63fa56373e3: Bootstrap complete.
I20260809 02:05:20.927531 18039 raft_consensus.cc:348] T 00000000000000000000000000000000 P a63f7b9159694eeebd13d63fa56373e3 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a63f7b9159694eeebd13d63fa56373e3" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 39909 } }
I20260809 02:05:20.928067 18039 raft_consensus.cc:729] T 00000000000000000000000000000000 P a63f7b9159694eeebd13d63fa56373e3 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: a63f7b9159694eeebd13d63fa56373e3, State: Initialized, Role: FOLLOWER
I20260809 02:05:20.928649 18039 consensus_queue.cc:260] T 00000000000000000000000000000000 P a63f7b9159694eeebd13d63fa56373e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a63f7b9159694eeebd13d63fa56373e3" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 39909 } }
I20260809 02:05:20.928798 18039 raft_consensus.cc:388] T 00000000000000000000000000000000 P a63f7b9159694eeebd13d63fa56373e3 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:05:20.928874 18039 raft_consensus.cc:482] T 00000000000000000000000000000000 P a63f7b9159694eeebd13d63fa56373e3 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:05:20.928973 18039 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a63f7b9159694eeebd13d63fa56373e3 [term 2 FOLLOWER]: Advancing to term 3
I20260809 02:05:20.931823 18039 raft_consensus.cc:504] T 00000000000000000000000000000000 P a63f7b9159694eeebd13d63fa56373e3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a63f7b9159694eeebd13d63fa56373e3" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 39909 } }
I20260809 02:05:20.932226 18039 leader_election.cc:302] T 00000000000000000000000000000000 P a63f7b9159694eeebd13d63fa56373e3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a63f7b9159694eeebd13d63fa56373e3; no voters:
I20260809 02:05:20.932605 18039 leader_election.cc:288] T 00000000000000000000000000000000 P a63f7b9159694eeebd13d63fa56373e3 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260809 02:05:20.932832 18044 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a63f7b9159694eeebd13d63fa56373e3 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 02:05:20.933149 18044 raft_consensus.cc:686] T 00000000000000000000000000000000 P a63f7b9159694eeebd13d63fa56373e3 [term 3 LEADER]: Becoming Leader. State: Replica: a63f7b9159694eeebd13d63fa56373e3, State: Running, Role: LEADER
I20260809 02:05:20.933591 18044 consensus_queue.cc:237] T 00000000000000000000000000000000 P a63f7b9159694eeebd13d63fa56373e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a63f7b9159694eeebd13d63fa56373e3" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 39909 } }
I20260809 02:05:20.934041 18039 sys_catalog.cc:564] T 00000000000000000000000000000000 P a63f7b9159694eeebd13d63fa56373e3 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:05:20.934999 18045 sys_catalog.cc:455] T 00000000000000000000000000000000 P a63f7b9159694eeebd13d63fa56373e3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "a63f7b9159694eeebd13d63fa56373e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a63f7b9159694eeebd13d63fa56373e3" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 39909 } } }
I20260809 02:05:20.935141 18045 sys_catalog.cc:458] T 00000000000000000000000000000000 P a63f7b9159694eeebd13d63fa56373e3 [sys.catalog]: This master's current role is: LEADER
I20260809 02:05:20.935623 18046 sys_catalog.cc:455] T 00000000000000000000000000000000 P a63f7b9159694eeebd13d63fa56373e3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a63f7b9159694eeebd13d63fa56373e3. Latest consensus state: current_term: 3 leader_uuid: "a63f7b9159694eeebd13d63fa56373e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a63f7b9159694eeebd13d63fa56373e3" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 39909 } } }
I20260809 02:05:20.935716 18046 sys_catalog.cc:458] T 00000000000000000000000000000000 P a63f7b9159694eeebd13d63fa56373e3 [sys.catalog]: This master's current role is: LEADER
I20260809 02:05:20.936157 18057 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:05:20.940596 18057 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:05:20.941191 18057 catalog_manager.cc:1259] Loaded cluster ID: 83f659d0a92b496f90e277950880f819
I20260809 02:05:20.941270 18057 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260809 02:05:20.942202 18057 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f83bb2e59ad google::LogMessage::Fail() at ??:0
@ 0x7f83bb2e888a google::LogMessage::SendToLog() at ??:0
@ 0x7f83bb2e54de google::LogMessage::Flush() at ??:0
@ 0x7f83bb2e6a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f83c5c24fdd _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f83c5c25d2f kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f83c5c22579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f83c5c66640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f83bd5373d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f83bd537c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f83bd5393f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f83bd528512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f83beb6c6db start_thread at ??:0
@ 0x7f83b9de361f clone at ??:0
W20260809 02:05:21.047390 18041 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:05:21.047820 18041 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:05:21.047892 18041 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:05:21.054162 18041 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:05:21.054258 18041 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:05:21.054352 18041 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.192.193
I20260809 02:05:21.062743 18041 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:39143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.193.keytab
--principal=kudu/127.1.192.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.192.193:44473
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.192.193
--webserver_port=36399
--tserver_master_addrs=127.1.192.254:39909
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18041
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.192.193
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:05:21.064280 18041 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:05:21.065719 18041 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:05:21.077185 18086 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:05:21.077164 18089 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:05:21.077322 18041 server_base.cc:1034] running on GCE node
W20260809 02:05:21.077170 18085 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:05:21.077986 18041 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:05:21.078704 18041 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:05:21.079905 18041 hybrid_clock.cc:648] HybridClock initialized: now 1786241121079891 us; error 39 us; skew 500 ppm
Aug 09 02:05:21 dist-test-slave-ss4n krb5kdc[16799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241121, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:05:21.083000 18041 init.cc:377] Logged in from keytab as kudu/127.1.192.193@KRBTEST.COM (short username kudu)
I20260809 02:05:21.083963 18041 webserver.cc:460] Webserver started at http://127.1.192.193:36399/ using document root <none> and password file <none>
I20260809 02:05:21.084564 18041 fs_manager.cc:359] Metadata directory not provided
I20260809 02:05:21.084661 18041 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:05:21.088919 18041 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260809 02:05:21.091653 18096 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:05:21.093132 18041 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 02:05:21.093315 18041 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-0/wal
uuid: "78569db4f20d4cfe884e53818df99efc"
format_stamp: "Formatted at 2026-08-09 02:05:18 on dist-test-slave-ss4n"
server_key: "ffa20ede60e9219240583a07f2b418c8"
server_key_iv: "2afde7e05377b354a77453b2d285e375"
server_key_version: "encryptionkey@0"
I20260809 02:05:21.094058 18041 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:05:21.125005 18041 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:05:21.128512 18041 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:05:21.128747 18041 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:05:21.129402 18041 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:05:21.130681 18103 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 02:05:21.134783 18041 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 02:05:21.134850 18041 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.004s user 0.001s sys 0.000s
I20260809 02:05:21.134898 18041 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 02:05:21.136580 18041 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 02:05:21.136644 18041 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260809 02:05:21.136888 18103 tablet_bootstrap.cc:492] T 29bdb8f0aa1e47dab98b2812c2da3908 P 78569db4f20d4cfe884e53818df99efc: Bootstrap starting.
I20260809 02:05:21.147893 18041 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.193:44473
I20260809 02:05:21.148144 18222 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.193:44473 every 8 connection(s)
I20260809 02:05:21.149171 18041 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-0/data/info.pb
I20260809 02:05:21.157544 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 18041
I20260809 02:05:21.158056 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.192.194:38717
--local_ip_for_outbound_sockets=127.1.192.194
--tserver_master_addrs=127.1.192.254:39909
--webserver_port=42009
--webserver_interface=127.1.192.194
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.194.keytab
--principal=kudu/127.1.192.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:39143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 02:05:21.158052 18103 log.cc:824] T 29bdb8f0aa1e47dab98b2812c2da3908 P 78569db4f20d4cfe884e53818df99efc: Log is configured to *not* fsync() on all Append() calls
I20260809 02:05:21.167742 18103 tablet_bootstrap.cc:492] T 29bdb8f0aa1e47dab98b2812c2da3908 P 78569db4f20d4cfe884e53818df99efc: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 02:05:21.168286 18103 tablet_bootstrap.cc:492] T 29bdb8f0aa1e47dab98b2812c2da3908 P 78569db4f20d4cfe884e53818df99efc: Bootstrap complete.
I20260809 02:05:21.169970 18103 ts_tablet_manager.cc:1397] T 29bdb8f0aa1e47dab98b2812c2da3908 P 78569db4f20d4cfe884e53818df99efc: Time spent bootstrapping tablet: real 0.033s user 0.021s sys 0.011s
I20260809 02:05:21.175424 18103 raft_consensus.cc:348] T 29bdb8f0aa1e47dab98b2812c2da3908 P 78569db4f20d4cfe884e53818df99efc [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce876b1fc4a840588948104f4e7eda5d" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38717 } } peers { permanent_uuid: "948a4d8653d148859d72dea2948ebb96" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 38931 } } peers { permanent_uuid: "78569db4f20d4cfe884e53818df99efc" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 44473 } }
I20260809 02:05:21.176021 18103 raft_consensus.cc:729] T 29bdb8f0aa1e47dab98b2812c2da3908 P 78569db4f20d4cfe884e53818df99efc [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78569db4f20d4cfe884e53818df99efc, State: Initialized, Role: FOLLOWER
I20260809 02:05:21.176584 18103 consensus_queue.cc:260] T 29bdb8f0aa1e47dab98b2812c2da3908 P 78569db4f20d4cfe884e53818df99efc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce876b1fc4a840588948104f4e7eda5d" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38717 } } peers { permanent_uuid: "948a4d8653d148859d72dea2948ebb96" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 38931 } } peers { permanent_uuid: "78569db4f20d4cfe884e53818df99efc" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 44473 } }
I20260809 02:05:21.177785 18103 ts_tablet_manager.cc:1428] T 29bdb8f0aa1e47dab98b2812c2da3908 P 78569db4f20d4cfe884e53818df99efc: Time spent starting tablet: real 0.008s user 0.006s sys 0.001s
W20260809 02:05:21.178907 18225 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:05:21.151578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:39909 (local address 127.1.192.193:57561)
0809 02:05:21.152237 (+ 659us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:21.152299 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0809 02:05:21.153316 (+ 1017us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:05:21.178113 (+ 24797us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:39909: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":416,"thread_start_us":209,"threads_started":1}
W20260809 02:05:21.179490 18223 heartbeater.cc:644] Failed to heartbeat to 127.1.192.254:39909 (0 consecutive failures): Network error: Failed to ping master at 127.1.192.254:39909: Client connection negotiation failed: client connection to 127.1.192.254:39909: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 02:05:21.180215 18225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:21.179957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:39909 (local address 127.1.192.193:43445)
0809 02:05:21.180079 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:21.180162 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:39909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260809 02:05:21.180943 18225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:21.180754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:39909 (local address 127.1.192.193:38885)
0809 02:05:21.180856 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:21.180898 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:39909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260809 02:05:21.181207 18223 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 02:05:21.281203 18227 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:05:21.281499 18227 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:05:21.281545 18227 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:05:21.285668 18227 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:05:21.285737 18227 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:05:21.285846 18227 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.192.194
I20260809 02:05:21.291478 18227 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:39143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.194.keytab
--principal=kudu/127.1.192.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.192.194:38717
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.192.194
--webserver_port=42009
--tserver_master_addrs=127.1.192.254:39909
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18227
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.192.194
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:05:21.292641 18227 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:05:21.293804 18227 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:05:21.303746 18235 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:05:21.303733 18237 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:05:21.303715 18234 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:05:21.304095 18227 server_base.cc:1034] running on GCE node
I20260809 02:05:21.304519 18227 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:05:21.305128 18227 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:05:21.306344 18227 hybrid_clock.cc:648] HybridClock initialized: now 1786241121306310 us; error 44 us; skew 500 ppm
Aug 09 02:05:21 dist-test-slave-ss4n krb5kdc[16799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241121, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:05:21.309230 18227 init.cc:377] Logged in from keytab as kudu/127.1.192.194@KRBTEST.COM (short username kudu)
I20260809 02:05:21.310240 18227 webserver.cc:460] Webserver started at http://127.1.192.194:42009/ using document root <none> and password file <none>
I20260809 02:05:21.310818 18227 fs_manager.cc:359] Metadata directory not provided
I20260809 02:05:21.310913 18227 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:05:21.314862 18227 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.003s
I20260809 02:05:21.317247 18244 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:05:21.318714 18227 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 02:05:21.318871 18227 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-1/wal
uuid: "ce876b1fc4a840588948104f4e7eda5d"
format_stamp: "Formatted at 2026-08-09 02:05:19 on dist-test-slave-ss4n"
server_key: "62739a957f5623802f673291825320ae"
server_key_iv: "22ee1ca0fb96c18830f71b6ffffcfede"
server_key_version: "encryptionkey@0"
I20260809 02:05:21.319525 18227 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:05:21.352588 18227 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:05:21.356202 18227 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:05:21.356465 18227 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:05:21.357143 18227 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:05:21.358088 18227 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:05:21.358168 18227 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:05:21.358237 18227 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:05:21.358289 18227 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:05:21.368079 18227 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.194:38717
I20260809 02:05:21.368124 18365 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.194:38717 every 8 connection(s)
I20260809 02:05:21.369202 18227 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-1/data/info.pb
I20260809 02:05:21.373018 18368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:21.371510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:39909 (local address 127.1.192.194:49353)
0809 02:05:21.372199 (+ 689us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:21.372312 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:39909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":120,"threads_started":1}
W20260809 02:05:21.373505 18366 heartbeater.cc:644] Failed to heartbeat to 127.1.192.254:39909 (0 consecutive failures): Network error: Failed to ping master at 127.1.192.254:39909: Client connection negotiation failed: client connection to 127.1.192.254:39909: connect: Connection refused (error 111)
I20260809 02:05:21.374157 18368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:21.373921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:39909 (local address 127.1.192.194:53875)
0809 02:05:21.374039 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:21.374103 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:39909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260809 02:05:21.374842 18368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:21.374649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:39909 (local address 127.1.192.194:56463)
0809 02:05:21.374751 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:21.374794 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:39909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260809 02:05:21.375048 18366 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 02:05:21.379083 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 18227
I20260809 02:05:21.379550 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.192.195:38931
--local_ip_for_outbound_sockets=127.1.192.195
--tserver_master_addrs=127.1.192.254:39909
--webserver_port=46035
--webserver_interface=127.1.192.195
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.195.keytab
--principal=kudu/127.1.192.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:39143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:05:21.499063 18369 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:05:21.499357 18369 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:05:21.499402 18369 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:05:21.503841 18369 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:05:21.503919 18369 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:05:21.503980 18369 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.192.195
I20260809 02:05:21.509294 18369 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:39143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.195.keytab
--principal=kudu/127.1.192.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.192.195:38931
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.192.195
--webserver_port=46035
--tserver_master_addrs=127.1.192.254:39909
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18369
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.192.195
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:05:21.510471 18369 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:05:21.511569 18369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:05:21.521734 18376 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:05:21.521734 18378 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:05:21.521742 18375 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:05:21.522185 18369 server_base.cc:1034] running on GCE node
I20260809 02:05:21.522671 18369 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:05:21.523451 18369 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:05:21.524662 18369 hybrid_clock.cc:648] HybridClock initialized: now 1786241121524630 us; error 42 us; skew 500 ppm
Aug 09 02:05:21 dist-test-slave-ss4n krb5kdc[16799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241121, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:05:21.527797 18369 init.cc:377] Logged in from keytab as kudu/127.1.192.195@KRBTEST.COM (short username kudu)
I20260809 02:05:21.528805 18369 webserver.cc:460] Webserver started at http://127.1.192.195:46035/ using document root <none> and password file <none>
I20260809 02:05:21.529412 18369 fs_manager.cc:359] Metadata directory not provided
I20260809 02:05:21.529537 18369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:05:21.533757 18369 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 02:05:21.535944 18385 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:05:21.537271 18369 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 02:05:21.537413 18369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-2/wal
uuid: "948a4d8653d148859d72dea2948ebb96"
format_stamp: "Formatted at 2026-08-09 02:05:19 on dist-test-slave-ss4n"
server_key: "644cd5db26089d8683d1c9b51441edd2"
server_key_iv: "b410d2bded5c3f09667fec9df33a7206"
server_key_version: "encryptionkey@0"
I20260809 02:05:21.538093 18369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:05:21.565680 18369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:05:21.569360 18369 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:05:21.569618 18369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:05:21.570299 18369 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:05:21.571218 18369 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:05:21.571296 18369 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:05:21.571375 18369 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:05:21.571422 18369 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:05:21.581352 18369 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.195:38931
I20260809 02:05:21.581403 18509 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.195:38931 every 8 connection(s)
I20260809 02:05:21.582551 18369 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786240941968412-1795-0/minicluster-data/ts-2/data/info.pb
I20260809 02:05:21.586599 18512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:21.584924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:39909 (local address 127.1.192.195:59883)
0809 02:05:21.585770 (+ 846us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:21.585884 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:39909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":641,"thread_start_us":137,"threads_started":1}
W20260809 02:05:21.587098 18510 heartbeater.cc:644] Failed to heartbeat to 127.1.192.254:39909 (0 consecutive failures): Network error: Failed to ping master at 127.1.192.254:39909: Client connection negotiation failed: client connection to 127.1.192.254:39909: connect: Connection refused (error 111)
I20260809 02:05:21.587809 18512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:21.587577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:39909 (local address 127.1.192.195:51969)
0809 02:05:21.587693 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:21.587759 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:39909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260809 02:05:21.588079 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 18369
I20260809 02:05:21.588660 18512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:21.588342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:39909 (local address 127.1.192.195:32813)
0809 02:05:21.588489 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:21.588607 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:39909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
W20260809 02:05:21.588907 18510 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 02:05:22.182428 18514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:22.181963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:39909 (local address 127.1.192.193:48887)
0809 02:05:22.182252 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:22.182314 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:39909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":93,"threads_started":1}
I20260809 02:05:22.376374 18515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:22.375874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:39909 (local address 127.1.192.194:33733)
0809 02:05:22.376194 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:22.376260 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:39909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":82,"threads_started":1}
I20260809 02:05:22.399180 18516 raft_consensus.cc:482] T 29bdb8f0aa1e47dab98b2812c2da3908 P 78569db4f20d4cfe884e53818df99efc [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:05:22.399441 18516 raft_consensus.cc:504] T 29bdb8f0aa1e47dab98b2812c2da3908 P 78569db4f20d4cfe884e53818df99efc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce876b1fc4a840588948104f4e7eda5d" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38717 } } peers { permanent_uuid: "948a4d8653d148859d72dea2948ebb96" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 38931 } } peers { permanent_uuid: "78569db4f20d4cfe884e53818df99efc" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 44473 } }
I20260809 02:05:22.400694 18516 leader_election.cc:288] T 29bdb8f0aa1e47dab98b2812c2da3908 P 78569db4f20d4cfe884e53818df99efc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ce876b1fc4a840588948104f4e7eda5d (127.1.192.194:38717), 948a4d8653d148859d72dea2948ebb96 (127.1.192.195:38931)
Aug 09 02:05:22 dist-test-slave-ss4n krb5kdc[16799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241121, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.193@KRBTEST.COM for kudu/127.1.192.194@KRBTEST.COM
Aug 09 02:05:22 dist-test-slave-ss4n krb5kdc[16799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241121, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.193@KRBTEST.COM for kudu/127.1.192.195@KRBTEST.COM
I20260809 02:05:22.418672 18519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:22.401169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.193:36537 (local address 127.1.192.194:38717)
0809 02:05:22.401619 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0809 02:05:22.401653 (+ 34us) server_negotiation.cc:399] Waiting for connection header
0809 02:05:22.401703 (+ 50us) server_negotiation.cc:407] Connection header received
0809 02:05:22.402719 (+ 1016us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:05:22.402729 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:05:22.403216 (+ 487us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:05:22.403586 (+ 370us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:05:22.405377 (+ 1791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:22.407204 (+ 1827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:22.409174 (+ 1970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:22.410032 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:22.410211 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:05:22.413856 (+ 3645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:05:22.413862 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:05:22.413874 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:05:22.413904 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:05:22.415869 (+ 1965us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:05:22.416473 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:05:22.416480 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:05:22.416485 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:05:22.416629 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:05:22.417000 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:05:22.417004 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:05:22.417006 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:05:22.417431 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:05:22.417617 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0809 02:05:22.418159 (+ 542us) server_negotiation.cc:299] Negotiation successful
0809 02:05:22.418458 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":327,"thread_start_us":79,"threads_started":1}
I20260809 02:05:22.418671 18514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:22.400975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.194:38717 (local address 127.1.192.193:36537)
0809 02:05:22.401252 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:22.401271 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 02:05:22.401381 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:05:22.403686 (+ 2305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:05:22.403693 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:05:22.404380 (+ 687us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:05:22.405143 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:05:22.405165 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:05:22.407400 (+ 2235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:05:22.407404 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:05:22.409023 (+ 1619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:05:22.409033 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:05:22.410184 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:05:22.410188 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:05:22.410526 (+ 338us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:05:22.411351 (+ 825us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:05:22.411375 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:05:22.413630 (+ 2255us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:05:22.416013 (+ 2383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:05:22.416020 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:05:22.416034 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:05:22.416326 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:05:22.416743 (+ 417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:05:22.416746 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:05:22.416748 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:05:22.416897 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:05:22.417594 (+ 697us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:05:22.417601 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:05:22.417831 (+ 230us) client_negotiation.cc:769] Sending connection context
0809 02:05:22.418040 (+ 209us) client_negotiation.cc:240] Negotiation successful
0809 02:05:22.418518 (+ 478us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"mutex_wait_us":451,"spinlock_wait_cycles":17280}
I20260809 02:05:22.418673 18517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:22.400975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.195:38931 (local address 127.1.192.193:36239)
0809 02:05:22.401621 (+ 646us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:22.401640 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 02:05:22.401764 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:05:22.403686 (+ 1922us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:05:22.403693 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:05:22.404380 (+ 687us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:05:22.405143 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:05:22.405165 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:05:22.407400 (+ 2235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:05:22.407404 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:05:22.409021 (+ 1617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:05:22.409033 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:05:22.410184 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:05:22.410188 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:05:22.410525 (+ 337us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:05:22.411365 (+ 840us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:05:22.411387 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:05:22.413969 (+ 2582us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:05:22.416460 (+ 2491us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:05:22.416465 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:05:22.416468 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:05:22.416709 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:05:22.417109 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:05:22.417114 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:05:22.417116 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:05:22.417214 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:05:22.417908 (+ 694us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:05:22.417912 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:05:22.418015 (+ 103us) client_negotiation.cc:769] Sending connection context
0809 02:05:22.418151 (+ 136us) client_negotiation.cc:240] Negotiation successful
0809 02:05:22.418436 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":576,"mutex_wait_us":443,"spinlock_wait_cycles":33408,"thread_start_us":83,"threads_started":1}
I20260809 02:05:22.419454 18518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:22.401168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.193:36239 (local address 127.1.192.195:38931)
0809 02:05:22.401701 (+ 533us) server_negotiation.cc:206] Beginning negotiation
0809 02:05:22.401715 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 02:05:22.401762 (+ 47us) server_negotiation.cc:407] Connection header received
0809 02:05:22.402731 (+ 969us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:05:22.402738 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:05:22.403216 (+ 478us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:05:22.403586 (+ 370us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:05:22.405377 (+ 1791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:22.407215 (+ 1838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:22.409174 (+ 1959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:22.410012 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:22.410210 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:05:22.414210 (+ 4000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:05:22.414222 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:05:22.414239 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:05:22.414282 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:05:22.416318 (+ 2036us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:05:22.416844 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:05:22.416851 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:05:22.416865 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:05:22.416989 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:05:22.417318 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:05:22.417323 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:05:22.417325 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:05:22.417789 (+ 464us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:05:22.417939 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0809 02:05:22.418295 (+ 356us) server_negotiation.cc:299] Negotiation successful
0809 02:05:22.418597 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":384,"thread_start_us":77,"threads_started":1}
W20260809 02:05:22.421451 18100 leader_election.cc:341] T 29bdb8f0aa1e47dab98b2812c2da3908 P 78569db4f20d4cfe884e53818df99efc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ce876b1fc4a840588948104f4e7eda5d (127.1.192.194:38717): Not found: Tablet not found: 29bdb8f0aa1e47dab98b2812c2da3908
W20260809 02:05:22.421665 18097 leader_election.cc:341] T 29bdb8f0aa1e47dab98b2812c2da3908 P 78569db4f20d4cfe884e53818df99efc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 948a4d8653d148859d72dea2948ebb96 (127.1.192.195:38931): Not found: Tablet not found: 29bdb8f0aa1e47dab98b2812c2da3908
I20260809 02:05:22.421826 18097 leader_election.cc:302] T 29bdb8f0aa1e47dab98b2812c2da3908 P 78569db4f20d4cfe884e53818df99efc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 78569db4f20d4cfe884e53818df99efc; no voters: 948a4d8653d148859d72dea2948ebb96, ce876b1fc4a840588948104f4e7eda5d
I20260809 02:05:22.422132 18516 raft_consensus.cc:2726] T 29bdb8f0aa1e47dab98b2812c2da3908 P 78569db4f20d4cfe884e53818df99efc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 02:05:22.590018 18520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:22.589581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:39909 (local address 127.1.192.195:38851)
0809 02:05:22.589855 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:22.589917 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:39909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":89,"threads_started":1}
I20260809 02:05:23.183696 18521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:23.183301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:39909 (local address 127.1.192.193:57343)
0809 02:05:23.183543 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:23.183601 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:39909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":81,"threads_started":1}
I20260809 02:05:23.377837 18522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:23.377312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:39909 (local address 127.1.192.194:43537)
0809 02:05:23.377679 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:23.377743 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:39909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":98,"threads_started":1}
I20260809 02:05:23.591540 18523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:23.590934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:39909 (local address 127.1.192.195:42151)
0809 02:05:23.591227 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:23.591292 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:39909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":115,"threads_started":1}
I20260809 02:05:24.185127 18524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:24.184606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:39909 (local address 127.1.192.193:47151)
0809 02:05:24.184954 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:24.185023 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:39909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":85,"threads_started":1}
I20260809 02:05:24.314594 18525 raft_consensus.cc:482] T 29bdb8f0aa1e47dab98b2812c2da3908 P 78569db4f20d4cfe884e53818df99efc [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:05:24.314733 18525 raft_consensus.cc:504] T 29bdb8f0aa1e47dab98b2812c2da3908 P 78569db4f20d4cfe884e53818df99efc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce876b1fc4a840588948104f4e7eda5d" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38717 } } peers { permanent_uuid: "948a4d8653d148859d72dea2948ebb96" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 38931 } } peers { permanent_uuid: "78569db4f20d4cfe884e53818df99efc" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 44473 } }
I20260809 02:05:24.315508 18525 leader_election.cc:288] T 29bdb8f0aa1e47dab98b2812c2da3908 P 78569db4f20d4cfe884e53818df99efc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ce876b1fc4a840588948104f4e7eda5d (127.1.192.194:38717), 948a4d8653d148859d72dea2948ebb96 (127.1.192.195:38931)
W20260809 02:05:24.316201 18097 leader_election.cc:341] T 29bdb8f0aa1e47dab98b2812c2da3908 P 78569db4f20d4cfe884e53818df99efc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 948a4d8653d148859d72dea2948ebb96 (127.1.192.195:38931): Not found: Tablet not found: 29bdb8f0aa1e47dab98b2812c2da3908
W20260809 02:05:24.316303 18100 leader_election.cc:341] T 29bdb8f0aa1e47dab98b2812c2da3908 P 78569db4f20d4cfe884e53818df99efc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ce876b1fc4a840588948104f4e7eda5d (127.1.192.194:38717): Not found: Tablet not found: 29bdb8f0aa1e47dab98b2812c2da3908
I20260809 02:05:24.316428 18100 leader_election.cc:302] T 29bdb8f0aa1e47dab98b2812c2da3908 P 78569db4f20d4cfe884e53818df99efc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 78569db4f20d4cfe884e53818df99efc; no voters: 948a4d8653d148859d72dea2948ebb96, ce876b1fc4a840588948104f4e7eda5d
I20260809 02:05:24.316637 18525 raft_consensus.cc:2726] T 29bdb8f0aa1e47dab98b2812c2da3908 P 78569db4f20d4cfe884e53818df99efc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 02:05:24.379209 18526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:24.378730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:39909 (local address 127.1.192.194:54071)
0809 02:05:24.379024 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:24.379096 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:39909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":80,"threads_started":1}
I20260809 02:05:24.592903 18527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:24.592439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:39909 (local address 127.1.192.195:57661)
0809 02:05:24.592746 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:24.592810 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:39909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":101,"threads_started":1}
I20260809 02:05:25.186384 18528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:25.185946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:39909 (local address 127.1.192.193:45181)
0809 02:05:25.186228 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:25.186287 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:39909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":81,"threads_started":1}
I20260809 02:05:25.380679 18529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:25.380112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:39909 (local address 127.1.192.194:60985)
0809 02:05:25.380429 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:25.380574 (+ 145us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:39909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":85,"threads_started":1}
I20260809 02:05:25.594254 18530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:25.593772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:39909 (local address 127.1.192.195:35091)
0809 02:05:25.594074 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:25.594137 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:39909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":103,"threads_started":1}
I20260809 02:05:26.188114 18531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:26.187487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:39909 (local address 127.1.192.193:51739)
0809 02:05:26.187927 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:26.187985 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:39909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":104,"threads_started":1}
I20260809 02:05:26.382205 18532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:26.381738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:39909 (local address 127.1.192.194:60501)
0809 02:05:26.382021 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:26.382087 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:39909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":86,"threads_started":1}
I20260809 02:05:26.595691 18533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:26.595151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:39909 (local address 127.1.192.195:53881)
0809 02:05:26.595448 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:26.595516 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:39909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":84,"threads_started":1}
I20260809 02:05:26.822136 18534 raft_consensus.cc:482] T 29bdb8f0aa1e47dab98b2812c2da3908 P 78569db4f20d4cfe884e53818df99efc [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:05:26.822302 18534 raft_consensus.cc:504] T 29bdb8f0aa1e47dab98b2812c2da3908 P 78569db4f20d4cfe884e53818df99efc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce876b1fc4a840588948104f4e7eda5d" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38717 } } peers { permanent_uuid: "948a4d8653d148859d72dea2948ebb96" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 38931 } } peers { permanent_uuid: "78569db4f20d4cfe884e53818df99efc" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 44473 } }
I20260809 02:05:26.822932 18534 leader_election.cc:288] T 29bdb8f0aa1e47dab98b2812c2da3908 P 78569db4f20d4cfe884e53818df99efc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ce876b1fc4a840588948104f4e7eda5d (127.1.192.194:38717), 948a4d8653d148859d72dea2948ebb96 (127.1.192.195:38931)
W20260809 02:05:26.823807 18100 leader_election.cc:341] T 29bdb8f0aa1e47dab98b2812c2da3908 P 78569db4f20d4cfe884e53818df99efc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ce876b1fc4a840588948104f4e7eda5d (127.1.192.194:38717): Not found: Tablet not found: 29bdb8f0aa1e47dab98b2812c2da3908
W20260809 02:05:26.823912 18097 leader_election.cc:341] T 29bdb8f0aa1e47dab98b2812c2da3908 P 78569db4f20d4cfe884e53818df99efc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 948a4d8653d148859d72dea2948ebb96 (127.1.192.195:38931): Not found: Tablet not found: 29bdb8f0aa1e47dab98b2812c2da3908
I20260809 02:05:26.823997 18097 leader_election.cc:302] T 29bdb8f0aa1e47dab98b2812c2da3908 P 78569db4f20d4cfe884e53818df99efc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 78569db4f20d4cfe884e53818df99efc; no voters: 948a4d8653d148859d72dea2948ebb96, ce876b1fc4a840588948104f4e7eda5d
I20260809 02:05:26.824165 18534 raft_consensus.cc:2726] T 29bdb8f0aa1e47dab98b2812c2da3908 P 78569db4f20d4cfe884e53818df99efc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 02:05:27.189594 18535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:27.189056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:39909 (local address 127.1.192.193:59353)
0809 02:05:27.189340 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:27.189424 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:39909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":107,"threads_started":1}
I20260809 02:05:27.383651 18537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:27.383127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:39909 (local address 127.1.192.194:36223)
0809 02:05:27.383442 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:27.383527 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:39909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":86,"threads_started":1}
I20260809 02:05:27.596983 18538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:27.596539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:39909 (local address 127.1.192.195:52305)
0809 02:05:27.596824 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:27.596885 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:39909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":89,"threads_started":1}
I20260809 02:05:28.190990 18539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:28.190566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:39909 (local address 127.1.192.193:33061)
0809 02:05:28.190840 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:28.190896 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:39909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":85,"threads_started":1}
I20260809 02:05:28.385140 18540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:28.384573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:39909 (local address 127.1.192.194:35807)
0809 02:05:28.384945 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:28.385023 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:39909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":79,"threads_started":1}
I20260809 02:05:28.598475 18541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:28.597980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:39909 (local address 127.1.192.195:33335)
0809 02:05:28.598318 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:28.598378 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:39909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":120,"threads_started":1}
I20260809 02:05:28.674224 18542 raft_consensus.cc:482] T 29bdb8f0aa1e47dab98b2812c2da3908 P 78569db4f20d4cfe884e53818df99efc [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:05:28.674403 18542 raft_consensus.cc:504] T 29bdb8f0aa1e47dab98b2812c2da3908 P 78569db4f20d4cfe884e53818df99efc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce876b1fc4a840588948104f4e7eda5d" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38717 } } peers { permanent_uuid: "948a4d8653d148859d72dea2948ebb96" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 38931 } } peers { permanent_uuid: "78569db4f20d4cfe884e53818df99efc" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 44473 } }
I20260809 02:05:28.674862 18542 leader_election.cc:288] T 29bdb8f0aa1e47dab98b2812c2da3908 P 78569db4f20d4cfe884e53818df99efc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ce876b1fc4a840588948104f4e7eda5d (127.1.192.194:38717), 948a4d8653d148859d72dea2948ebb96 (127.1.192.195:38931)
W20260809 02:05:28.675786 18100 leader_election.cc:341] T 29bdb8f0aa1e47dab98b2812c2da3908 P 78569db4f20d4cfe884e53818df99efc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ce876b1fc4a840588948104f4e7eda5d (127.1.192.194:38717): Not found: Tablet not found: 29bdb8f0aa1e47dab98b2812c2da3908
W20260809 02:05:28.675897 18097 leader_election.cc:341] T 29bdb8f0aa1e47dab98b2812c2da3908 P 78569db4f20d4cfe884e53818df99efc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 948a4d8653d148859d72dea2948ebb96 (127.1.192.195:38931): Not found: Tablet not found: 29bdb8f0aa1e47dab98b2812c2da3908
I20260809 02:05:28.675971 18097 leader_election.cc:302] T 29bdb8f0aa1e47dab98b2812c2da3908 P 78569db4f20d4cfe884e53818df99efc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 78569db4f20d4cfe884e53818df99efc; no voters: 948a4d8653d148859d72dea2948ebb96, ce876b1fc4a840588948104f4e7eda5d
I20260809 02:05:28.676227 18542 raft_consensus.cc:2726] T 29bdb8f0aa1e47dab98b2812c2da3908 P 78569db4f20d4cfe884e53818df99efc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 02:05:29.192399 18543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:29.191972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:39909 (local address 127.1.192.193:39663)
0809 02:05:29.192243 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:29.192299 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:39909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":87,"threads_started":1}
I20260809 02:05:29.386796 18544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:29.386273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:39909 (local address 127.1.192.194:48431)
0809 02:05:29.386614 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:29.386688 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:39909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":84,"threads_started":1}
I20260809 02:05:29.600157 18545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:29.599611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:39909 (local address 127.1.192.195:33301)
0809 02:05:29.599975 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:29.600057 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:39909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":126,"threads_started":1}
I20260809 02:05:30.193910 18546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:30.193311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:39909 (local address 127.1.192.193:48763)
0809 02:05:30.193737 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:30.193803 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:39909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":101,"threads_started":1}
I20260809 02:05:30.388417 18547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:30.387847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:39909 (local address 127.1.192.194:53561)
0809 02:05:30.388243 (+ 396us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:30.388315 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:39909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":114,"threads_started":1}
I20260809 02:05:30.601576 18548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:30.601016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:39909 (local address 127.1.192.195:35215)
0809 02:05:30.601335 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:30.601399 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:39909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":87,"threads_started":1}
I20260809 02:05:31.195307 18549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:31.194846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:39909 (local address 127.1.192.193:44747)
0809 02:05:31.195117 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:31.195210 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:39909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":118,"threads_started":1}
I20260809 02:05:31.389926 18552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:31.389437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:39909 (local address 127.1.192.194:51671)
0809 02:05:31.389761 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:31.389827 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:39909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":113,"threads_started":1}
I20260809 02:05:31.602887 18553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:31.602464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:39909 (local address 127.1.192.195:48555)
0809 02:05:31.602740 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:31.602793 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:39909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":119,"threads_started":1}
I20260809 02:05:32.197000 18554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:32.196313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:39909 (local address 127.1.192.193:48107)
0809 02:05:32.196769 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:32.196853 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:39909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":114,"threads_started":1}
I20260809 02:05:32.391271 18555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:32.390812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:39909 (local address 127.1.192.194:55025)
0809 02:05:32.391110 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:32.391175 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:39909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":92,"threads_started":1}
I20260809 02:05:32.604175 18556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:32.603727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:39909 (local address 127.1.192.195:57511)
0809 02:05:32.604012 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:32.604078 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:39909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":107,"threads_started":1}
I20260809 02:05:32.642732 18557 raft_consensus.cc:482] T 29bdb8f0aa1e47dab98b2812c2da3908 P 78569db4f20d4cfe884e53818df99efc [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:05:32.642923 18557 raft_consensus.cc:504] T 29bdb8f0aa1e47dab98b2812c2da3908 P 78569db4f20d4cfe884e53818df99efc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce876b1fc4a840588948104f4e7eda5d" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38717 } } peers { permanent_uuid: "948a4d8653d148859d72dea2948ebb96" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 38931 } } peers { permanent_uuid: "78569db4f20d4cfe884e53818df99efc" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 44473 } }
I20260809 02:05:32.643522 18557 leader_election.cc:288] T 29bdb8f0aa1e47dab98b2812c2da3908 P 78569db4f20d4cfe884e53818df99efc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ce876b1fc4a840588948104f4e7eda5d (127.1.192.194:38717), 948a4d8653d148859d72dea2948ebb96 (127.1.192.195:38931)
W20260809 02:05:32.644554 18097 leader_election.cc:341] T 29bdb8f0aa1e47dab98b2812c2da3908 P 78569db4f20d4cfe884e53818df99efc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 948a4d8653d148859d72dea2948ebb96 (127.1.192.195:38931): Not found: Tablet not found: 29bdb8f0aa1e47dab98b2812c2da3908
W20260809 02:05:32.644673 18100 leader_election.cc:341] T 29bdb8f0aa1e47dab98b2812c2da3908 P 78569db4f20d4cfe884e53818df99efc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ce876b1fc4a840588948104f4e7eda5d (127.1.192.194:38717): Not found: Tablet not found: 29bdb8f0aa1e47dab98b2812c2da3908
I20260809 02:05:32.644768 18100 leader_election.cc:302] T 29bdb8f0aa1e47dab98b2812c2da3908 P 78569db4f20d4cfe884e53818df99efc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 78569db4f20d4cfe884e53818df99efc; no voters: 948a4d8653d148859d72dea2948ebb96, ce876b1fc4a840588948104f4e7eda5d
I20260809 02:05:32.644932 18557 raft_consensus.cc:2726] T 29bdb8f0aa1e47dab98b2812c2da3908 P 78569db4f20d4cfe884e53818df99efc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 02:05:33.198547 18558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:33.198018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:39909 (local address 127.1.192.193:40643)
0809 02:05:33.198374 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:33.198443 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:39909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":83,"threads_started":1}
I20260809 02:05:33.392709 18559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:33.392199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:39909 (local address 127.1.192.194:52345)
0809 02:05:33.392488 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:33.392553 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:39909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":108,"threads_started":1}
I20260809 02:05:33.605669 18560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:33.605053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:39909 (local address 127.1.192.195:41905)
0809 02:05:33.605371 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:33.605450 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:39909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":99,"threads_started":1}
I20260809 02:05:34.199985 18561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:34.199454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:39909 (local address 127.1.192.193:46783)
0809 02:05:34.199804 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:34.199871 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:39909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":118,"threads_started":1}
I20260809 02:05:34.394169 18562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:34.393690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:39909 (local address 127.1.192.194:54753)
0809 02:05:34.393966 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:34.394066 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:39909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":75,"threads_started":1}
I20260809 02:05:34.606997 18563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:34.606516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:39909 (local address 127.1.192.195:46753)
0809 02:05:34.606824 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:34.606895 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:39909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":96,"threads_started":1}
I20260809 02:05:35.201436 18564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:35.200898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:39909 (local address 127.1.192.193:43303)
0809 02:05:35.201258 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:35.201327 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:39909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":127,"threads_started":1}
I20260809 02:05:35.395470 18565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:35.395028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:39909 (local address 127.1.192.194:43727)
0809 02:05:35.395306 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:35.395363 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:39909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":92,"threads_started":1}
I20260809 02:05:35.608340 18566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:35.607874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:39909 (local address 127.1.192.195:49701)
0809 02:05:35.608169 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:35.608239 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:39909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":116,"threads_started":1}
I20260809 02:05:36.203111 18567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:36.202637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:39909 (local address 127.1.192.193:58405)
0809 02:05:36.202919 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:36.202984 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:39909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":76,"threads_started":1}
I20260809 02:05:36.396993 18568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:36.396521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:39909 (local address 127.1.192.194:52269)
0809 02:05:36.396824 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:36.396885 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:39909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":96,"threads_started":1}
I20260809 02:05:36.594467 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 18041
I20260809 02:05:36.602823 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 18227
I20260809 02:05:36.609335 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 18369
2026-08-09T02:05:36Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (19570 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:05:36 dist-test-slave-ss4n krb5kdc[18571](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:05:36 dist-test-slave-ss4n krb5kdc[18571](info): set up 2 sockets
Aug 09 02:05:36 dist-test-slave-ss4n krb5kdc[18571](info): commencing operation
krb5kdc: starting...
W20260809 02:05:37.722357 1795 mini_kdc.cc:121] Time spent starting KDC: real 1.086s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/krb5kdc/test-user.keytab.
Aug 09 02:05:37 dist-test-slave-ss4n krb5kdc[18571](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241137, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:05:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:05:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.192.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.192.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.192.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab.
Entry for principal kudu/127.1.192.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab.
I20260809 02:05:37.866422 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.192.254:42427
--webserver_interface=127.1.192.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab
--principal=kudu/127.1.192.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:33273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.192.254:42427
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:05:37.986557 18585 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:05:37.986897 18585 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:05:37.986974 18585 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:05:37.991536 18585 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 02:05:37.991643 18585 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 02:05:37.991674 18585 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 02:05:37.991701 18585 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:05:37.991750 18585 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:05:37.991802 18585 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:05:37.991843 18585 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:05:37.991886 18585 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:05:37.998229 18585 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:33273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.1.192.254:42427
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab
--principal=kudu/127.1.192.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.192.254:42427
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.192.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18585
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:05:37.999430 18585 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:05:38.000557 18585 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:05:38.009737 18592 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:05:38.009876 18585 server_base.cc:1034] running on GCE node
W20260809 02:05:38.009684 18594 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:05:38.009769 18591 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:05:38.010532 18585 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:05:38.011977 18585 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:05:38.013170 18585 hybrid_clock.cc:648] HybridClock initialized: now 1786241138013122 us; error 59 us; skew 500 ppm
Aug 09 02:05:38 dist-test-slave-ss4n krb5kdc[18571](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241138, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:05:38.016301 18585 init.cc:377] Logged in from keytab as kudu/127.1.192.254@KRBTEST.COM (short username kudu)
I20260809 02:05:38.017423 18585 webserver.cc:460] Webserver started at http://127.1.192.254:40169/ using document root <none> and password file <none>
I20260809 02:05:38.018069 18585 fs_manager.cc:359] Metadata directory not provided
I20260809 02:05:38.018168 18585 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:05:38.018415 18585 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:05:38.020687 18585 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/master-0/data/instance:
uuid: "9d388ae00b2c47b78dd7ca03332c3071"
format_stamp: "Formatted at 2026-08-09 02:05:38 on dist-test-slave-ss4n"
server_key: "531e65c25971eaa04e81ad921adb7ed0"
server_key_iv: "e6e3375925835d2446c298332cab4ecd"
server_key_version: "encryptionkey@0"
I20260809 02:05:38.021312 18585 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/master-0/wal/instance:
uuid: "9d388ae00b2c47b78dd7ca03332c3071"
format_stamp: "Formatted at 2026-08-09 02:05:38 on dist-test-slave-ss4n"
server_key: "531e65c25971eaa04e81ad921adb7ed0"
server_key_iv: "e6e3375925835d2446c298332cab4ecd"
server_key_version: "encryptionkey@0"
I20260809 02:05:38.025332 18585 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.003s
I20260809 02:05:38.028174 18603 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:05:38.029691 18585 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 02:05:38.029832 18585 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/master-0/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/master-0/wal
uuid: "9d388ae00b2c47b78dd7ca03332c3071"
format_stamp: "Formatted at 2026-08-09 02:05:38 on dist-test-slave-ss4n"
server_key: "531e65c25971eaa04e81ad921adb7ed0"
server_key_iv: "e6e3375925835d2446c298332cab4ecd"
server_key_version: "encryptionkey@0"
I20260809 02:05:38.029953 18585 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:05:38.052250 18585 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:05:38.055585 18585 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:05:38.055817 18585 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:05:38.063426 18585 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.254:42427
I20260809 02:05:38.063783 18658 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.254:42427 every 8 connection(s)
I20260809 02:05:38.064491 18585 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
I20260809 02:05:38.064824 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 18585
I20260809 02:05:38.064957 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/master-0/wal/instance
I20260809 02:05:38.065349 1795 external_mini_cluster.cc:1428] Setting key 79344fe8735bc08a64ab87b830f154fa
I20260809 02:05:38.067672 18659 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:05:38.074628 18659 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d388ae00b2c47b78dd7ca03332c3071: Bootstrap starting.
Aug 09 02:05:38 dist-test-slave-ss4n krb5kdc[18571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241137, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.192.254@KRBTEST.COM
I20260809 02:05:38.077241 18659 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9d388ae00b2c47b78dd7ca03332c3071: Neither blocks nor log segments found. Creating new log.
I20260809 02:05:38.078073 18659 log.cc:824] T 00000000000000000000000000000000 P 9d388ae00b2c47b78dd7ca03332c3071: Log is configured to *not* fsync() on all Append() calls
I20260809 02:05:38.080591 18659 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d388ae00b2c47b78dd7ca03332c3071: No bootstrap required, opened a new log
I20260809 02:05:38.081862 18662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:38.067084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41200 (local address 127.1.192.254:42427)
0809 02:05:38.067547 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0809 02:05:38.067565 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0809 02:05:38.067604 (+ 39us) server_negotiation.cc:407] Connection header received
0809 02:05:38.068405 (+ 801us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:05:38.068424 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:05:38.068779 (+ 355us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:05:38.069103 (+ 324us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:05:38.070125 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:38.071196 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:38.072393 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:38.072850 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:38.072966 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:05:38.076251 (+ 3285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:05:38.076256 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:05:38.076269 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:05:38.076301 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:05:38.078516 (+ 2215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:05:38.079010 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:05:38.079015 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:05:38.079020 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:05:38.079136 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:05:38.079550 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:05:38.079565 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:05:38.079567 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:05:38.080088 (+ 521us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:05:38.080267 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0809 02:05:38.080590 (+ 323us) server_negotiation.cc:299] Negotiation successful
0809 02:05:38.080965 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":299,"thread_start_us":152,"threads_started":1}
I20260809 02:05:38.083907 18659 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9d388ae00b2c47b78dd7ca03332c3071 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d388ae00b2c47b78dd7ca03332c3071" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 42427 } }
I20260809 02:05:38.084164 18659 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9d388ae00b2c47b78dd7ca03332c3071 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:05:38.084249 18659 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9d388ae00b2c47b78dd7ca03332c3071 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d388ae00b2c47b78dd7ca03332c3071, State: Initialized, Role: FOLLOWER
I20260809 02:05:38.084770 18659 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9d388ae00b2c47b78dd7ca03332c3071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d388ae00b2c47b78dd7ca03332c3071" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 42427 } }
I20260809 02:05:38.084903 18659 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9d388ae00b2c47b78dd7ca03332c3071 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:05:38.084945 18659 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9d388ae00b2c47b78dd7ca03332c3071 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:05:38.085017 18659 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9d388ae00b2c47b78dd7ca03332c3071 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:05:38.086016 18659 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9d388ae00b2c47b78dd7ca03332c3071 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d388ae00b2c47b78dd7ca03332c3071" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 42427 } }
I20260809 02:05:38.086323 18659 leader_election.cc:302] T 00000000000000000000000000000000 P 9d388ae00b2c47b78dd7ca03332c3071 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9d388ae00b2c47b78dd7ca03332c3071; no voters:
I20260809 02:05:38.086630 18659 leader_election.cc:288] T 00000000000000000000000000000000 P 9d388ae00b2c47b78dd7ca03332c3071 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:05:38.086712 18664 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9d388ae00b2c47b78dd7ca03332c3071 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:05:38.086992 18664 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9d388ae00b2c47b78dd7ca03332c3071 [term 1 LEADER]: Becoming Leader. State: Replica: 9d388ae00b2c47b78dd7ca03332c3071, State: Running, Role: LEADER
I20260809 02:05:38.087324 18664 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9d388ae00b2c47b78dd7ca03332c3071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d388ae00b2c47b78dd7ca03332c3071" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 42427 } }
I20260809 02:05:38.088001 18659 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9d388ae00b2c47b78dd7ca03332c3071 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:05:38.089112 18666 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d388ae00b2c47b78dd7ca03332c3071 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9d388ae00b2c47b78dd7ca03332c3071. Latest consensus state: current_term: 1 leader_uuid: "9d388ae00b2c47b78dd7ca03332c3071" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d388ae00b2c47b78dd7ca03332c3071" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 42427 } } }
I20260809 02:05:38.089355 18666 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d388ae00b2c47b78dd7ca03332c3071 [sys.catalog]: This master's current role is: LEADER
I20260809 02:05:38.089444 18665 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d388ae00b2c47b78dd7ca03332c3071 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9d388ae00b2c47b78dd7ca03332c3071" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d388ae00b2c47b78dd7ca03332c3071" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 42427 } } }
I20260809 02:05:38.089587 18665 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d388ae00b2c47b78dd7ca03332c3071 [sys.catalog]: This master's current role is: LEADER
W20260809 02:05:38.092885 18681 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9d388ae00b2c47b78dd7ca03332c3071: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 02:05:38.092957 18681 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 02:05:38.093050 18673 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:05:38.093851 18673 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:05:38.099064 18673 catalog_manager.cc:1347] Generated new cluster ID: 7ae31931097d4e48b0910208ee0e9f08
I20260809 02:05:38.099138 18673 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:05:38.114243 18673 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:05:38.115343 18673 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:05:38.130820 18673 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9d388ae00b2c47b78dd7ca03332c3071: Generated new TSK 0
I20260809 02:05:38.131698 18673 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.192.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.192.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.192.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.193.keytab.
Entry for principal kudu/127.1.192.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.193.keytab.
I20260809 02:05:38.178471 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.192.193:0
--local_ip_for_outbound_sockets=127.1.192.193
--webserver_interface=127.1.192.193
--webserver_port=0
--tserver_master_addrs=127.1.192.254:42427
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.193.keytab
--principal=kudu/127.1.192.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:33273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:05:38.294802 18687 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:05:38.295143 18687 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:05:38.295226 18687 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:05:38.299392 18687 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:05:38.299515 18687 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:05:38.299618 18687 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.192.193
I20260809 02:05:38.305184 18687 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:33273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.193.keytab
--principal=kudu/127.1.192.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.192.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.192.193
--webserver_port=0
--tserver_master_addrs=127.1.192.254:42427
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18687
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.192.193
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:05:38.306398 18687 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:05:38.307482 18687 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:05:38.317655 18693 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:05:38.317664 18694 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:05:38.317790 18687 server_base.cc:1034] running on GCE node
W20260809 02:05:38.317664 18696 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:05:38.318375 18687 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:05:38.319092 18687 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:05:38.320288 18687 hybrid_clock.cc:648] HybridClock initialized: now 1786241138320269 us; error 34 us; skew 500 ppm
Aug 09 02:05:38 dist-test-slave-ss4n krb5kdc[18571](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241138, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:05:38.323565 18687 init.cc:377] Logged in from keytab as kudu/127.1.192.193@KRBTEST.COM (short username kudu)
I20260809 02:05:38.324501 18687 webserver.cc:460] Webserver started at http://127.1.192.193:37133/ using document root <none> and password file <none>
I20260809 02:05:38.325040 18687 fs_manager.cc:359] Metadata directory not provided
I20260809 02:05:38.325105 18687 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:05:38.325275 18687 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:05:38.327374 18687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-0/data/instance:
uuid: "1a567ef48489496e8e8a33b6a9329eda"
format_stamp: "Formatted at 2026-08-09 02:05:38 on dist-test-slave-ss4n"
server_key: "194b68000e2f13498410a1257ea231ce"
server_key_iv: "5f1024baeffe9f33765e4e4ad219ca64"
server_key_version: "encryptionkey@0"
I20260809 02:05:38.327911 18687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-0/wal/instance:
uuid: "1a567ef48489496e8e8a33b6a9329eda"
format_stamp: "Formatted at 2026-08-09 02:05:38 on dist-test-slave-ss4n"
server_key: "194b68000e2f13498410a1257ea231ce"
server_key_iv: "5f1024baeffe9f33765e4e4ad219ca64"
server_key_version: "encryptionkey@0"
I20260809 02:05:38.331852 18687 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 02:05:38.334513 18703 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:05:38.335844 18687 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 02:05:38.335979 18687 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-0/wal
uuid: "1a567ef48489496e8e8a33b6a9329eda"
format_stamp: "Formatted at 2026-08-09 02:05:38 on dist-test-slave-ss4n"
server_key: "194b68000e2f13498410a1257ea231ce"
server_key_iv: "5f1024baeffe9f33765e4e4ad219ca64"
server_key_version: "encryptionkey@0"
I20260809 02:05:38.336112 18687 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:05:38.348694 18687 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:05:38.351997 18687 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:05:38.352213 18687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:05:38.352846 18687 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:05:38.353750 18687 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:05:38.353821 18687 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:05:38.353900 18687 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:05:38.353960 18687 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:05:38.363509 18687 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.193:46211
I20260809 02:05:38.363531 18825 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.193:46211 every 8 connection(s)
I20260809 02:05:38.364461 18687 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-0/data/info.pb
I20260809 02:05:38.366904 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 18687
I20260809 02:05:38.367048 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-0/wal/instance
I20260809 02:05:38.367390 1795 external_mini_cluster.cc:1428] Setting key 3361422a24053963ae3a8b0f54881be4
Aug 09 02:05:38 dist-test-slave-ss4n krb5kdc[18571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241138, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.193@KRBTEST.COM for kudu/127.1.192.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.192.194@KRBTEST.COM; defaulting to no policy
I20260809 02:05:38.379868 18662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:38.366455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.193:43505 (local address 127.1.192.254:42427)
0809 02:05:38.366643 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0809 02:05:38.366647 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:05:38.368012 (+ 1365us) server_negotiation.cc:407] Connection header received
0809 02:05:38.369090 (+ 1078us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:05:38.369094 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:05:38.369150 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:05:38.369241 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:05:38.370558 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:38.371154 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:38.372460 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:38.372725 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:38.372851 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:05:38.376000 (+ 3149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:05:38.376004 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:05:38.376007 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:05:38.376042 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:05:38.377707 (+ 1665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:05:38.378254 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:05:38.378258 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:05:38.378260 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:05:38.378365 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:05:38.378766 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:05:38.378768 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:05:38.378769 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:05:38.378957 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:05:38.379036 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 02:05:38.379533 (+ 497us) server_negotiation.cc:299] Negotiation successful
0809 02:05:38.379741 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.192.194@KRBTEST.COM" created.
I20260809 02:05:38.380951 18828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:38.366886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:42427 (local address 127.1.192.193:43505)
0809 02:05:38.367787 (+ 901us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:38.367854 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0809 02:05:38.368804 (+ 950us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:05:38.369418 (+ 614us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:05:38.369425 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:05:38.369890 (+ 465us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:05:38.370398 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:05:38.370412 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:05:38.371294 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:05:38.371298 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:05:38.372309 (+ 1011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:05:38.372319 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:05:38.372898 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:05:38.372902 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:05:38.373107 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:05:38.373870 (+ 763us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:05:38.373894 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:05:38.375788 (+ 1894us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:05:38.377832 (+ 2044us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:05:38.377838 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:05:38.377855 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:05:38.378142 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:05:38.378489 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:05:38.378491 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:05:38.378493 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:05:38.378666 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:05:38.379059 (+ 393us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:05:38.379064 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:05:38.379314 (+ 250us) client_negotiation.cc:769] Sending connection context
0809 02:05:38.379478 (+ 164us) client_negotiation.cc:240] Negotiation successful
0809 02:05:38.379846 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":665,"thread_start_us":157,"threads_started":1}
I20260809 02:05:38.382302 18826 heartbeater.cc:347] Connected to a master server at 127.1.192.254:42427
I20260809 02:05:38.382578 18826 heartbeater.cc:464] Registering TS with master...
I20260809 02:05:38.383178 18826 heartbeater.cc:511] Master 127.1.192.254:42427 requested a full tablet report, sending...
I20260809 02:05:38.384861 18622 ts_manager.cc:194] Registered new tserver with Master: 1a567ef48489496e8e8a33b6a9329eda (127.1.192.193:46211)
I20260809 02:05:38.386065 18622 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.192.193@KRBTEST.COM'} at 127.1.192.193:43505
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.192.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.194.keytab.
Entry for principal kudu/127.1.192.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.194.keytab.
I20260809 02:05:38.400979 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.192.194:0
--local_ip_for_outbound_sockets=127.1.192.194
--webserver_interface=127.1.192.194
--webserver_port=0
--tserver_master_addrs=127.1.192.254:42427
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.194.keytab
--principal=kudu/127.1.192.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:33273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:05:38.517449 18832 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:05:38.517802 18832 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:05:38.517849 18832 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:05:38.521960 18832 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:05:38.522020 18832 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:05:38.522089 18832 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.192.194
I20260809 02:05:38.527359 18832 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:33273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.194.keytab
--principal=kudu/127.1.192.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.192.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.192.194
--webserver_port=0
--tserver_master_addrs=127.1.192.254:42427
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18832
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.192.194
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:05:38.528509 18832 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:05:38.529630 18832 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:05:38.540320 18837 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:05:38.540330 18838 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:05:38.540378 18832 server_base.cc:1034] running on GCE node
W20260809 02:05:38.540330 18840 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:05:38.541077 18832 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:05:38.541848 18832 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:05:38.543021 18832 hybrid_clock.cc:648] HybridClock initialized: now 1786241138543003 us; error 42 us; skew 500 ppm
Aug 09 02:05:38 dist-test-slave-ss4n krb5kdc[18571](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241138, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:05:38.546093 18832 init.cc:377] Logged in from keytab as kudu/127.1.192.194@KRBTEST.COM (short username kudu)
I20260809 02:05:38.547025 18832 webserver.cc:460] Webserver started at http://127.1.192.194:36429/ using document root <none> and password file <none>
I20260809 02:05:38.547621 18832 fs_manager.cc:359] Metadata directory not provided
I20260809 02:05:38.547716 18832 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:05:38.547961 18832 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:05:38.550189 18832 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-1/data/instance:
uuid: "e8a36744066c43debf42fc9fa6d47efd"
format_stamp: "Formatted at 2026-08-09 02:05:38 on dist-test-slave-ss4n"
server_key: "8214780cf82e86dfdcc54772a3e6d20a"
server_key_iv: "c4fe70c99d60c3a0f7122d7260b3a121"
server_key_version: "encryptionkey@0"
I20260809 02:05:38.550797 18832 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-1/wal/instance:
uuid: "e8a36744066c43debf42fc9fa6d47efd"
format_stamp: "Formatted at 2026-08-09 02:05:38 on dist-test-slave-ss4n"
server_key: "8214780cf82e86dfdcc54772a3e6d20a"
server_key_iv: "c4fe70c99d60c3a0f7122d7260b3a121"
server_key_version: "encryptionkey@0"
I20260809 02:05:38.554723 18832 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260809 02:05:38.557363 18847 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:05:38.558826 18832 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 02:05:38.558933 18832 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-1/wal
uuid: "e8a36744066c43debf42fc9fa6d47efd"
format_stamp: "Formatted at 2026-08-09 02:05:38 on dist-test-slave-ss4n"
server_key: "8214780cf82e86dfdcc54772a3e6d20a"
server_key_iv: "c4fe70c99d60c3a0f7122d7260b3a121"
server_key_version: "encryptionkey@0"
I20260809 02:05:38.559033 18832 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:05:38.569845 18832 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:05:38.573220 18832 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:05:38.573393 18832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:05:38.574204 18832 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:05:38.575070 18832 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:05:38.575124 18832 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:05:38.575168 18832 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:05:38.575186 18832 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:05:38.585035 18832 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.194:38117
I20260809 02:05:38.585117 18968 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.194:38117 every 8 connection(s)
I20260809 02:05:38.586146 18832 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-1/data/info.pb
I20260809 02:05:38.588805 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 18832
I20260809 02:05:38.588930 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-1/wal/instance
I20260809 02:05:38.589344 1795 external_mini_cluster.cc:1428] Setting key a83e5226d204acf5f6ef6d5889ccf820
Aug 09 02:05:38 dist-test-slave-ss4n krb5kdc[18571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241138, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.194@KRBTEST.COM for kudu/127.1.192.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.192.195@KRBTEST.COM; defaulting to no policy
I20260809 02:05:38.603758 18662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:38.588376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.194:45265 (local address 127.1.192.254:42427)
0809 02:05:38.588541 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0809 02:05:38.588546 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:05:38.590320 (+ 1774us) server_negotiation.cc:407] Connection header received
0809 02:05:38.591457 (+ 1137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:05:38.591459 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:05:38.591506 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:05:38.591581 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:05:38.593205 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:38.594200 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:38.595906 (+ 1706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:38.596186 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:38.596299 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:05:38.599395 (+ 3096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:05:38.599398 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:05:38.599400 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:05:38.599426 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:05:38.601224 (+ 1798us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:05:38.601941 (+ 717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:05:38.601945 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:05:38.601948 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:05:38.602091 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:05:38.602512 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:05:38.602516 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:05:38.602518 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:05:38.602728 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:05:38.602850 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0809 02:05:38.603430 (+ 580us) server_negotiation.cc:299] Negotiation successful
0809 02:05:38.603577 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.192.195@KRBTEST.COM" created.
I20260809 02:05:38.605018 18972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:38.588806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:42427 (local address 127.1.192.194:45265)
0809 02:05:38.590124 (+ 1318us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:38.590212 (+ 88us) client_negotiation.cc:174] Beginning negotiation
0809 02:05:38.591230 (+ 1018us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:05:38.591759 (+ 529us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:05:38.591770 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:05:38.592323 (+ 553us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:05:38.593025 (+ 702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:05:38.593046 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:05:38.594338 (+ 1292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:05:38.594341 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:05:38.595746 (+ 1405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:05:38.595759 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:05:38.596371 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:05:38.596378 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:05:38.596626 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:05:38.597351 (+ 725us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:05:38.597378 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:05:38.599203 (+ 1825us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:05:38.601375 (+ 2172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:05:38.601381 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:05:38.601398 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:05:38.601816 (+ 418us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:05:38.602211 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:05:38.602215 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:05:38.602217 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:05:38.602391 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:05:38.602850 (+ 459us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:05:38.602859 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:05:38.603141 (+ 282us) client_negotiation.cc:769] Sending connection context
0809 02:05:38.603351 (+ 210us) client_negotiation.cc:240] Negotiation successful
0809 02:05:38.603825 (+ 474us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1045,"thread_start_us":155,"threads_started":1}
I20260809 02:05:38.606626 18970 heartbeater.cc:347] Connected to a master server at 127.1.192.254:42427
I20260809 02:05:38.606905 18970 heartbeater.cc:464] Registering TS with master...
I20260809 02:05:38.607442 18970 heartbeater.cc:511] Master 127.1.192.254:42427 requested a full tablet report, sending...
I20260809 02:05:38.608610 18622 ts_manager.cc:194] Registered new tserver with Master: e8a36744066c43debf42fc9fa6d47efd (127.1.192.194:38117)
I20260809 02:05:38.609292 18622 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.192.194@KRBTEST.COM'} at 127.1.192.194:45265
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.192.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.195.keytab.
Entry for principal kudu/127.1.192.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.195.keytab.
I20260809 02:05:38.626434 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.192.195:0
--local_ip_for_outbound_sockets=127.1.192.195
--webserver_interface=127.1.192.195
--webserver_port=0
--tserver_master_addrs=127.1.192.254:42427
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.195.keytab
--principal=kudu/127.1.192.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:33273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:05:38.745704 18975 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:05:38.746014 18975 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:05:38.746124 18975 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:05:38.750247 18975 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:05:38.750344 18975 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:05:38.750434 18975 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.192.195
I20260809 02:05:38.755858 18975 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:33273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.195.keytab
--principal=kudu/127.1.192.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.192.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.192.195
--webserver_port=0
--tserver_master_addrs=127.1.192.254:42427
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18975
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.192.195
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:05:38.756980 18975 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:05:38.758064 18975 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:05:38.767990 18983 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:05:38.768129 18981 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:05:38.767973 18980 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:05:38.768157 18975 server_base.cc:1034] running on GCE node
I20260809 02:05:38.768818 18975 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:05:38.769570 18975 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:05:38.770754 18975 hybrid_clock.cc:648] HybridClock initialized: now 1786241138770720 us; error 48 us; skew 500 ppm
Aug 09 02:05:38 dist-test-slave-ss4n krb5kdc[18571](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241138, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:05:38.773654 18975 init.cc:377] Logged in from keytab as kudu/127.1.192.195@KRBTEST.COM (short username kudu)
I20260809 02:05:38.774544 18975 webserver.cc:460] Webserver started at http://127.1.192.195:41081/ using document root <none> and password file <none>
I20260809 02:05:38.775136 18975 fs_manager.cc:359] Metadata directory not provided
I20260809 02:05:38.775233 18975 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:05:38.775449 18975 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:05:38.777994 18975 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-2/data/instance:
uuid: "f4d44812f3f84c4db2c91ba76af08a89"
format_stamp: "Formatted at 2026-08-09 02:05:38 on dist-test-slave-ss4n"
server_key: "9bdb0b28e7ab8d3776b4a2c64ecaba1d"
server_key_iv: "6f139f9db5bf58e0ea8fd14d29fb5ccd"
server_key_version: "encryptionkey@0"
I20260809 02:05:38.778656 18975 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-2/wal/instance:
uuid: "f4d44812f3f84c4db2c91ba76af08a89"
format_stamp: "Formatted at 2026-08-09 02:05:38 on dist-test-slave-ss4n"
server_key: "9bdb0b28e7ab8d3776b4a2c64ecaba1d"
server_key_iv: "6f139f9db5bf58e0ea8fd14d29fb5ccd"
server_key_version: "encryptionkey@0"
I20260809 02:05:38.782939 18975 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 02:05:38.785750 18990 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:05:38.787055 18975 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 02:05:38.787210 18975 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-2/wal
uuid: "f4d44812f3f84c4db2c91ba76af08a89"
format_stamp: "Formatted at 2026-08-09 02:05:38 on dist-test-slave-ss4n"
server_key: "9bdb0b28e7ab8d3776b4a2c64ecaba1d"
server_key_iv: "6f139f9db5bf58e0ea8fd14d29fb5ccd"
server_key_version: "encryptionkey@0"
I20260809 02:05:38.787338 18975 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:05:38.817211 18975 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:05:38.820606 18975 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:05:38.820837 18975 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:05:38.821463 18975 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:05:38.822417 18975 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:05:38.822467 18975 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:05:38.822537 18975 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:05:38.822570 18975 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:05:38.832238 18975 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.195:40857
I20260809 02:05:38.832302 19110 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.195:40857 every 8 connection(s)
I20260809 02:05:38.833207 18975 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-2/data/info.pb
I20260809 02:05:38.835557 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 18975
I20260809 02:05:38.835718 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-2/wal/instance
I20260809 02:05:38.836071 1795 external_mini_cluster.cc:1428] Setting key b1f12102cd81a71d5c9e88ec64e09037
Aug 09 02:05:38 dist-test-slave-ss4n krb5kdc[18571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241138, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.195@KRBTEST.COM for kudu/127.1.192.254@KRBTEST.COM
I20260809 02:05:38.848635 18662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:38.835318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.195:38591 (local address 127.1.192.254:42427)
0809 02:05:38.835481 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0809 02:05:38.835487 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:05:38.836738 (+ 1251us) server_negotiation.cc:407] Connection header received
0809 02:05:38.837904 (+ 1166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:05:38.837908 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:05:38.837992 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:05:38.838091 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:05:38.839480 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:38.840243 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:38.841396 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:38.841756 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:38.841851 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:05:38.844523 (+ 2672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:05:38.844526 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:05:38.844528 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:05:38.844555 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:05:38.846252 (+ 1697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:05:38.846980 (+ 728us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:05:38.846984 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:05:38.846986 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:05:38.847121 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:05:38.847552 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:05:38.847555 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:05:38.847556 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:05:38.847759 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:05:38.847850 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 02:05:38.848369 (+ 519us) server_negotiation.cc:299] Negotiation successful
0809 02:05:38.848492 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 02:05:38.849675 19114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:38.835769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:42427 (local address 127.1.192.195:38591)
0809 02:05:38.836544 (+ 775us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:38.836593 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 02:05:38.837658 (+ 1065us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:05:38.838233 (+ 575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:05:38.838240 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:05:38.838813 (+ 573us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:05:38.839313 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:05:38.839350 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:05:38.840366 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:05:38.840369 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:05:38.841274 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:05:38.841283 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:05:38.841901 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:05:38.841904 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:05:38.842054 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:05:38.842679 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:05:38.842695 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:05:38.844347 (+ 1652us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:05:38.846396 (+ 2049us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:05:38.846402 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:05:38.846419 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:05:38.846837 (+ 418us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:05:38.847276 (+ 439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:05:38.847278 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:05:38.847280 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:05:38.847416 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:05:38.847861 (+ 445us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:05:38.847867 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:05:38.848118 (+ 251us) client_negotiation.cc:769] Sending connection context
0809 02:05:38.848282 (+ 164us) client_negotiation.cc:240] Negotiation successful
0809 02:05:38.848546 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":557,"thread_start_us":131,"threads_started":1}
I20260809 02:05:38.850927 19111 heartbeater.cc:347] Connected to a master server at 127.1.192.254:42427
I20260809 02:05:38.851179 19111 heartbeater.cc:464] Registering TS with master...
I20260809 02:05:38.851663 19111 heartbeater.cc:511] Master 127.1.192.254:42427 requested a full tablet report, sending...
I20260809 02:05:38.852689 18622 ts_manager.cc:194] Registered new tserver with Master: f4d44812f3f84c4db2c91ba76af08a89 (127.1.192.195:40857)
I20260809 02:05:38.853271 18622 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.192.195@KRBTEST.COM'} at 127.1.192.195:38591
I20260809 02:05:38.863170 1795 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 02:05:38.874154 18662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:38.865472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41212 (local address 127.1.192.254:42427)
0809 02:05:38.865665 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0809 02:05:38.865669 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:05:38.866003 (+ 334us) server_negotiation.cc:407] Connection header received
0809 02:05:38.866077 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:05:38.866079 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:05:38.866124 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:05:38.866197 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:05:38.867225 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:38.867875 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:38.868912 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:38.869171 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:38.869262 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:05:38.871051 (+ 1789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:05:38.871055 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:05:38.871058 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:05:38.871083 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:05:38.872417 (+ 1334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:05:38.872856 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:05:38.872859 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:05:38.872860 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:05:38.872940 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:05:38.873210 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:05:38.873212 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:05:38.873213 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:05:38.873435 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:05:38.873587 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0809 02:05:38.873906 (+ 319us) server_negotiation.cc:299] Negotiation successful
0809 02:05:38.874011 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 02:05:38.877666 18622 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41212:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 02:05:38.880952 18622 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:05:38.897558 19123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:38.890730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.194:38117 (local address 127.0.0.1:41064)
0809 02:05:38.891775 (+ 1045us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:38.891829 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 02:05:38.891959 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:05:38.892678 (+ 719us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:05:38.892688 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:05:38.892716 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:05:38.892893 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:05:38.892904 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:05:38.894645 (+ 1741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:05:38.894651 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:05:38.895828 (+ 1177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:05:38.895838 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:05:38.896892 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:05:38.896895 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:05:38.897024 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:05:38.897082 (+ 58us) client_negotiation.cc:769] Sending connection context
0809 02:05:38.897196 (+ 114us) client_negotiation.cc:240] Negotiation successful
0809 02:05:38.897336 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":857,"mutex_wait_us":64,"thread_start_us":137,"threads_started":1}
I20260809 02:05:38.897854 19126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:38.891473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41064 (local address 127.1.192.194:38117)
0809 02:05:38.891947 (+ 474us) server_negotiation.cc:206] Beginning negotiation
0809 02:05:38.891959 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 02:05:38.891982 (+ 23us) server_negotiation.cc:407] Connection header received
0809 02:05:38.892304 (+ 322us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:05:38.892312 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:05:38.892513 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:05:38.892649 (+ 136us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:05:38.893026 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:38.894324 (+ 1298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:38.895959 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:38.896747 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:38.896859 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:05:38.897008 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0809 02:05:38.897596 (+ 588us) server_negotiation.cc:299] Negotiation successful
0809 02:05:38.897709 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":369,"thread_start_us":159,"threads_started":1}
I20260809 02:05:38.897912 19128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:38.891685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45322 (local address 127.1.192.195:40857)
0809 02:05:38.893061 (+ 1376us) server_negotiation.cc:206] Beginning negotiation
0809 02:05:38.893080 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0809 02:05:38.893097 (+ 17us) server_negotiation.cc:407] Connection header received
0809 02:05:38.893155 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:05:38.893158 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:05:38.893282 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:05:38.893431 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:05:38.893675 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:38.894463 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:38.896320 (+ 1857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:38.896926 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:38.897029 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:05:38.897166 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0809 02:05:38.897665 (+ 499us) server_negotiation.cc:299] Negotiation successful
0809 02:05:38.897777 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1265,"thread_start_us":94,"threads_started":1}
I20260809 02:05:38.898082 19125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:38.891034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38724 (local address 127.1.192.193:46211)
0809 02:05:38.891936 (+ 902us) server_negotiation.cc:206] Beginning negotiation
0809 02:05:38.891943 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 02:05:38.892328 (+ 385us) server_negotiation.cc:407] Connection header received
0809 02:05:38.892414 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:05:38.892421 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:05:38.892624 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:05:38.892774 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:05:38.893106 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:38.894289 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:38.896299 (+ 2010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:38.897098 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:38.897217 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:05:38.897352 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0809 02:05:38.897660 (+ 308us) server_negotiation.cc:299] Negotiation successful
0809 02:05:38.897919 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":801,"thread_start_us":139,"threads_started":1}
I20260809 02:05:38.898229 19127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:38.891547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.195:40857 (local address 127.0.0.1:45322)
0809 02:05:38.892124 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:38.892138 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 02:05:38.892235 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:05:38.893409 (+ 1174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:05:38.893412 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:05:38.893430 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:05:38.893555 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:05:38.893563 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:05:38.894603 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:05:38.894606 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:05:38.896161 (+ 1555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:05:38.896172 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:05:38.897228 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:05:38.897231 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:05:38.897372 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:05:38.897410 (+ 38us) client_negotiation.cc:769] Sending connection context
0809 02:05:38.897497 (+ 87us) client_negotiation.cc:240] Negotiation successful
0809 02:05:38.897576 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":510,"mutex_wait_us":192,"spinlock_wait_cycles":7424,"thread_start_us":108,"threads_started":1}
I20260809 02:05:38.898643 19124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:38.890730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.193:46211 (local address 127.0.0.1:38724)
0809 02:05:38.892097 (+ 1367us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:38.892111 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 02:05:38.892229 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:05:38.892772 (+ 543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:05:38.892775 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:05:38.892796 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:05:38.892939 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:05:38.892944 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:05:38.894461 (+ 1517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:05:38.894464 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:05:38.896161 (+ 1697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:05:38.896172 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:05:38.897230 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:05:38.897234 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:05:38.897425 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:05:38.897450 (+ 25us) client_negotiation.cc:769] Sending connection context
0809 02:05:38.898421 (+ 971us) client_negotiation.cc:240] Negotiation successful
0809 02:05:38.898501 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1278,"mutex_wait_us":451,"spinlock_wait_cycles":2304,"thread_start_us":83,"threads_started":1}
I20260809 02:05:38.901196 18752 tablet_service.cc:1467] Processing CreateTablet for tablet 679fccce20264d8aad85bf8c30572606 (DEFAULT_TABLE table=test-table [id=e86faf1482d4457fba4341034dabc7ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:05:38.901278 19040 tablet_service.cc:1467] Processing CreateTablet for tablet 679fccce20264d8aad85bf8c30572606 (DEFAULT_TABLE table=test-table [id=e86faf1482d4457fba4341034dabc7ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:05:38.901279 18899 tablet_service.cc:1467] Processing CreateTablet for tablet 679fccce20264d8aad85bf8c30572606 (DEFAULT_TABLE table=test-table [id=e86faf1482d4457fba4341034dabc7ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:05:38.902346 18752 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 679fccce20264d8aad85bf8c30572606. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:05:38.902371 19040 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 679fccce20264d8aad85bf8c30572606. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:05:38.902366 18899 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 679fccce20264d8aad85bf8c30572606. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:05:38.909009 19129 tablet_bootstrap.cc:492] T 679fccce20264d8aad85bf8c30572606 P 1a567ef48489496e8e8a33b6a9329eda: Bootstrap starting.
I20260809 02:05:38.910431 19130 tablet_bootstrap.cc:492] T 679fccce20264d8aad85bf8c30572606 P e8a36744066c43debf42fc9fa6d47efd: Bootstrap starting.
I20260809 02:05:38.911831 19129 tablet_bootstrap.cc:654] T 679fccce20264d8aad85bf8c30572606 P 1a567ef48489496e8e8a33b6a9329eda: Neither blocks nor log segments found. Creating new log.
I20260809 02:05:38.912549 19130 tablet_bootstrap.cc:654] T 679fccce20264d8aad85bf8c30572606 P e8a36744066c43debf42fc9fa6d47efd: Neither blocks nor log segments found. Creating new log.
I20260809 02:05:38.912606 19129 log.cc:824] T 679fccce20264d8aad85bf8c30572606 P 1a567ef48489496e8e8a33b6a9329eda: Log is configured to *not* fsync() on all Append() calls
I20260809 02:05:38.913333 19131 tablet_bootstrap.cc:492] T 679fccce20264d8aad85bf8c30572606 P f4d44812f3f84c4db2c91ba76af08a89: Bootstrap starting.
I20260809 02:05:38.913357 19130 log.cc:824] T 679fccce20264d8aad85bf8c30572606 P e8a36744066c43debf42fc9fa6d47efd: Log is configured to *not* fsync() on all Append() calls
I20260809 02:05:38.915086 19130 tablet_bootstrap.cc:492] T 679fccce20264d8aad85bf8c30572606 P e8a36744066c43debf42fc9fa6d47efd: No bootstrap required, opened a new log
I20260809 02:05:38.915138 19129 tablet_bootstrap.cc:492] T 679fccce20264d8aad85bf8c30572606 P 1a567ef48489496e8e8a33b6a9329eda: No bootstrap required, opened a new log
I20260809 02:05:38.915310 19130 ts_tablet_manager.cc:1397] T 679fccce20264d8aad85bf8c30572606 P e8a36744066c43debf42fc9fa6d47efd: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260809 02:05:38.915314 19129 ts_tablet_manager.cc:1397] T 679fccce20264d8aad85bf8c30572606 P 1a567ef48489496e8e8a33b6a9329eda: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260809 02:05:38.916103 19131 tablet_bootstrap.cc:654] T 679fccce20264d8aad85bf8c30572606 P f4d44812f3f84c4db2c91ba76af08a89: Neither blocks nor log segments found. Creating new log.
I20260809 02:05:38.916944 19131 log.cc:824] T 679fccce20264d8aad85bf8c30572606 P f4d44812f3f84c4db2c91ba76af08a89: Log is configured to *not* fsync() on all Append() calls
I20260809 02:05:38.918661 19130 raft_consensus.cc:348] T 679fccce20264d8aad85bf8c30572606 P e8a36744066c43debf42fc9fa6d47efd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a36744066c43debf42fc9fa6d47efd" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38117 } } peers { permanent_uuid: "1a567ef48489496e8e8a33b6a9329eda" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 46211 } } peers { permanent_uuid: "f4d44812f3f84c4db2c91ba76af08a89" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 40857 } }
I20260809 02:05:38.918927 19130 raft_consensus.cc:374] T 679fccce20264d8aad85bf8c30572606 P e8a36744066c43debf42fc9fa6d47efd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:05:38.919000 19130 raft_consensus.cc:729] T 679fccce20264d8aad85bf8c30572606 P e8a36744066c43debf42fc9fa6d47efd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8a36744066c43debf42fc9fa6d47efd, State: Initialized, Role: FOLLOWER
I20260809 02:05:38.919286 19131 tablet_bootstrap.cc:492] T 679fccce20264d8aad85bf8c30572606 P f4d44812f3f84c4db2c91ba76af08a89: No bootstrap required, opened a new log
I20260809 02:05:38.919490 19131 ts_tablet_manager.cc:1397] T 679fccce20264d8aad85bf8c30572606 P f4d44812f3f84c4db2c91ba76af08a89: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260809 02:05:38.919518 19130 consensus_queue.cc:260] T 679fccce20264d8aad85bf8c30572606 P e8a36744066c43debf42fc9fa6d47efd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a36744066c43debf42fc9fa6d47efd" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38117 } } peers { permanent_uuid: "1a567ef48489496e8e8a33b6a9329eda" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 46211 } } peers { permanent_uuid: "f4d44812f3f84c4db2c91ba76af08a89" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 40857 } }
I20260809 02:05:38.920234 18970 heartbeater.cc:503] Master 127.1.192.254:42427 was elected leader, sending a full tablet report...
I20260809 02:05:38.920606 19130 ts_tablet_manager.cc:1428] T 679fccce20264d8aad85bf8c30572606 P e8a36744066c43debf42fc9fa6d47efd: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260809 02:05:38.921293 19129 raft_consensus.cc:348] T 679fccce20264d8aad85bf8c30572606 P 1a567ef48489496e8e8a33b6a9329eda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a36744066c43debf42fc9fa6d47efd" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38117 } } peers { permanent_uuid: "1a567ef48489496e8e8a33b6a9329eda" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 46211 } } peers { permanent_uuid: "f4d44812f3f84c4db2c91ba76af08a89" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 40857 } }
I20260809 02:05:38.921597 19129 raft_consensus.cc:374] T 679fccce20264d8aad85bf8c30572606 P 1a567ef48489496e8e8a33b6a9329eda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:05:38.921700 19129 raft_consensus.cc:729] T 679fccce20264d8aad85bf8c30572606 P 1a567ef48489496e8e8a33b6a9329eda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a567ef48489496e8e8a33b6a9329eda, State: Initialized, Role: FOLLOWER
I20260809 02:05:38.922142 19129 consensus_queue.cc:260] T 679fccce20264d8aad85bf8c30572606 P 1a567ef48489496e8e8a33b6a9329eda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a36744066c43debf42fc9fa6d47efd" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38117 } } peers { permanent_uuid: "1a567ef48489496e8e8a33b6a9329eda" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 46211 } } peers { permanent_uuid: "f4d44812f3f84c4db2c91ba76af08a89" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 40857 } }
I20260809 02:05:38.922865 18826 heartbeater.cc:503] Master 127.1.192.254:42427 was elected leader, sending a full tablet report...
I20260809 02:05:38.923386 19131 raft_consensus.cc:348] T 679fccce20264d8aad85bf8c30572606 P f4d44812f3f84c4db2c91ba76af08a89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a36744066c43debf42fc9fa6d47efd" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38117 } } peers { permanent_uuid: "1a567ef48489496e8e8a33b6a9329eda" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 46211 } } peers { permanent_uuid: "f4d44812f3f84c4db2c91ba76af08a89" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 40857 } }
I20260809 02:05:38.923645 19131 raft_consensus.cc:374] T 679fccce20264d8aad85bf8c30572606 P f4d44812f3f84c4db2c91ba76af08a89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:05:38.923703 19131 raft_consensus.cc:729] T 679fccce20264d8aad85bf8c30572606 P f4d44812f3f84c4db2c91ba76af08a89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4d44812f3f84c4db2c91ba76af08a89, State: Initialized, Role: FOLLOWER
I20260809 02:05:38.923740 19129 ts_tablet_manager.cc:1428] T 679fccce20264d8aad85bf8c30572606 P 1a567ef48489496e8e8a33b6a9329eda: Time spent starting tablet: real 0.008s user 0.008s sys 0.000s
I20260809 02:05:38.924224 19131 consensus_queue.cc:260] T 679fccce20264d8aad85bf8c30572606 P f4d44812f3f84c4db2c91ba76af08a89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a36744066c43debf42fc9fa6d47efd" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38117 } } peers { permanent_uuid: "1a567ef48489496e8e8a33b6a9329eda" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 46211 } } peers { permanent_uuid: "f4d44812f3f84c4db2c91ba76af08a89" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 40857 } }
I20260809 02:05:38.924889 19111 heartbeater.cc:503] Master 127.1.192.254:42427 was elected leader, sending a full tablet report...
I20260809 02:05:38.926175 19131 ts_tablet_manager.cc:1428] T 679fccce20264d8aad85bf8c30572606 P f4d44812f3f84c4db2c91ba76af08a89: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
W20260809 02:05:39.084492 19112 tablet.cc:2369] T 679fccce20264d8aad85bf8c30572606 P f4d44812f3f84c4db2c91ba76af08a89: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 02:05:39.087674 18971 tablet.cc:2369] T 679fccce20264d8aad85bf8c30572606 P e8a36744066c43debf42fc9fa6d47efd: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 02:05:39.116045 18827 tablet.cc:2369] T 679fccce20264d8aad85bf8c30572606 P 1a567ef48489496e8e8a33b6a9329eda: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 02:05:39.295252 19135 raft_consensus.cc:482] T 679fccce20264d8aad85bf8c30572606 P e8a36744066c43debf42fc9fa6d47efd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:05:39.295529 19135 raft_consensus.cc:504] T 679fccce20264d8aad85bf8c30572606 P e8a36744066c43debf42fc9fa6d47efd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a36744066c43debf42fc9fa6d47efd" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38117 } } peers { permanent_uuid: "1a567ef48489496e8e8a33b6a9329eda" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 46211 } } peers { permanent_uuid: "f4d44812f3f84c4db2c91ba76af08a89" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 40857 } }
I20260809 02:05:39.296730 19135 leader_election.cc:288] T 679fccce20264d8aad85bf8c30572606 P e8a36744066c43debf42fc9fa6d47efd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1a567ef48489496e8e8a33b6a9329eda (127.1.192.193:46211), f4d44812f3f84c4db2c91ba76af08a89 (127.1.192.195:40857)
I20260809 02:05:39.300884 19139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:39.297094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.195:40857 (local address 127.1.192.194:33385)
0809 02:05:39.297662 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:39.297675 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:05:39.297830 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:05:39.298095 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:05:39.298098 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:05:39.298115 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:05:39.298243 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:05:39.298248 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:05:39.298970 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:05:39.298975 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:05:39.299785 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:05:39.299794 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:05:39.300427 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:05:39.300431 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:05:39.300531 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:05:39.300559 (+ 28us) client_negotiation.cc:769] Sending connection context
0809 02:05:39.300654 (+ 95us) client_negotiation.cc:240] Negotiation successful
0809 02:05:39.300724 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":461,"mutex_wait_us":151,"thread_start_us":67,"threads_started":1}
I20260809 02:05:39.300864 19128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:39.297353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.194:33385 (local address 127.1.192.195:40857)
0809 02:05:39.297471 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0809 02:05:39.297476 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:05:39.297776 (+ 300us) server_negotiation.cc:407] Connection header received
0809 02:05:39.297939 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:05:39.297942 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:05:39.297992 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:05:39.298084 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:05:39.298356 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:39.298859 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:39.299912 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:39.300301 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:39.300430 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:05:39.300501 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 02:05:39.300669 (+ 168us) server_negotiation.cc:299] Negotiation successful
0809 02:05:39.300739 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 02:05:39.301277 19125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:39.297257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.194:50259 (local address 127.1.192.193:46211)
0809 02:05:39.297406 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0809 02:05:39.297411 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:05:39.297760 (+ 349us) server_negotiation.cc:407] Connection header received
0809 02:05:39.297964 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:05:39.297968 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:05:39.298026 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:05:39.298126 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:05:39.298443 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:39.298989 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:39.300185 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:39.300662 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:39.300811 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:05:39.300863 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 02:05:39.301071 (+ 208us) server_negotiation.cc:299] Negotiation successful
0809 02:05:39.301163 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 02:05:39.301328 19138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:39.297094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.193:46211 (local address 127.1.192.194:50259)
0809 02:05:39.297628 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:39.297644 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 02:05:39.297802 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:05:39.298138 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:05:39.298142 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:05:39.298160 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:05:39.298308 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:05:39.298314 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:05:39.299127 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:05:39.299132 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:05:39.300032 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:05:39.300041 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:05:39.300803 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:05:39.300807 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:05:39.300918 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:05:39.300950 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 02:05:39.301044 (+ 94us) client_negotiation.cc:240] Negotiation successful
0809 02:05:39.301139 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":98,"threads_started":1}
I20260809 02:05:39.301842 19061 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "679fccce20264d8aad85bf8c30572606" candidate_uuid: "e8a36744066c43debf42fc9fa6d47efd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4d44812f3f84c4db2c91ba76af08a89" is_pre_election: true
I20260809 02:05:39.302057 18775 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "679fccce20264d8aad85bf8c30572606" candidate_uuid: "e8a36744066c43debf42fc9fa6d47efd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a567ef48489496e8e8a33b6a9329eda" is_pre_election: true
I20260809 02:05:39.302253 19061 raft_consensus.cc:2454] T 679fccce20264d8aad85bf8c30572606 P f4d44812f3f84c4db2c91ba76af08a89 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8a36744066c43debf42fc9fa6d47efd in term 0.
I20260809 02:05:39.302357 18775 raft_consensus.cc:2454] T 679fccce20264d8aad85bf8c30572606 P 1a567ef48489496e8e8a33b6a9329eda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8a36744066c43debf42fc9fa6d47efd in term 0.
I20260809 02:05:39.302855 18850 leader_election.cc:302] T 679fccce20264d8aad85bf8c30572606 P e8a36744066c43debf42fc9fa6d47efd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 1a567ef48489496e8e8a33b6a9329eda, e8a36744066c43debf42fc9fa6d47efd, f4d44812f3f84c4db2c91ba76af08a89; no voters:
I20260809 02:05:39.303203 19135 raft_consensus.cc:2781] T 679fccce20264d8aad85bf8c30572606 P e8a36744066c43debf42fc9fa6d47efd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:05:39.303287 19135 raft_consensus.cc:482] T 679fccce20264d8aad85bf8c30572606 P e8a36744066c43debf42fc9fa6d47efd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:05:39.303318 19135 raft_consensus.cc:3037] T 679fccce20264d8aad85bf8c30572606 P e8a36744066c43debf42fc9fa6d47efd [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:05:39.304414 19135 raft_consensus.cc:504] T 679fccce20264d8aad85bf8c30572606 P e8a36744066c43debf42fc9fa6d47efd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a36744066c43debf42fc9fa6d47efd" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38117 } } peers { permanent_uuid: "1a567ef48489496e8e8a33b6a9329eda" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 46211 } } peers { permanent_uuid: "f4d44812f3f84c4db2c91ba76af08a89" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 40857 } }
I20260809 02:05:39.304790 19135 leader_election.cc:288] T 679fccce20264d8aad85bf8c30572606 P e8a36744066c43debf42fc9fa6d47efd [CANDIDATE]: Term 1 election: Requested vote from peers 1a567ef48489496e8e8a33b6a9329eda (127.1.192.193:46211), f4d44812f3f84c4db2c91ba76af08a89 (127.1.192.195:40857)
I20260809 02:05:39.305184 18775 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "679fccce20264d8aad85bf8c30572606" candidate_uuid: "e8a36744066c43debf42fc9fa6d47efd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a567ef48489496e8e8a33b6a9329eda"
I20260809 02:05:39.305310 19061 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "679fccce20264d8aad85bf8c30572606" candidate_uuid: "e8a36744066c43debf42fc9fa6d47efd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4d44812f3f84c4db2c91ba76af08a89"
I20260809 02:05:39.305322 18775 raft_consensus.cc:3037] T 679fccce20264d8aad85bf8c30572606 P 1a567ef48489496e8e8a33b6a9329eda [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:05:39.305562 19061 raft_consensus.cc:3037] T 679fccce20264d8aad85bf8c30572606 P f4d44812f3f84c4db2c91ba76af08a89 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:05:39.306543 18775 raft_consensus.cc:2454] T 679fccce20264d8aad85bf8c30572606 P 1a567ef48489496e8e8a33b6a9329eda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8a36744066c43debf42fc9fa6d47efd in term 1.
I20260809 02:05:39.306885 18851 leader_election.cc:302] T 679fccce20264d8aad85bf8c30572606 P e8a36744066c43debf42fc9fa6d47efd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a567ef48489496e8e8a33b6a9329eda, e8a36744066c43debf42fc9fa6d47efd; no voters:
I20260809 02:05:39.306988 19061 raft_consensus.cc:2454] T 679fccce20264d8aad85bf8c30572606 P f4d44812f3f84c4db2c91ba76af08a89 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8a36744066c43debf42fc9fa6d47efd in term 1.
I20260809 02:05:39.307047 19135 raft_consensus.cc:2781] T 679fccce20264d8aad85bf8c30572606 P e8a36744066c43debf42fc9fa6d47efd [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:05:39.307299 19135 raft_consensus.cc:686] T 679fccce20264d8aad85bf8c30572606 P e8a36744066c43debf42fc9fa6d47efd [term 1 LEADER]: Becoming Leader. State: Replica: e8a36744066c43debf42fc9fa6d47efd, State: Running, Role: LEADER
I20260809 02:05:39.307644 19135 consensus_queue.cc:237] T 679fccce20264d8aad85bf8c30572606 P e8a36744066c43debf42fc9fa6d47efd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a36744066c43debf42fc9fa6d47efd" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38117 } } peers { permanent_uuid: "1a567ef48489496e8e8a33b6a9329eda" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 46211 } } peers { permanent_uuid: "f4d44812f3f84c4db2c91ba76af08a89" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 40857 } }
I20260809 02:05:39.311192 18618 catalog_manager.cc:5495] T 679fccce20264d8aad85bf8c30572606 P e8a36744066c43debf42fc9fa6d47efd reported cstate change: term changed from 0 to 1, leader changed from <none> to e8a36744066c43debf42fc9fa6d47efd (127.1.192.194). New cstate: current_term: 1 leader_uuid: "e8a36744066c43debf42fc9fa6d47efd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a36744066c43debf42fc9fa6d47efd" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38117 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1a567ef48489496e8e8a33b6a9329eda" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 46211 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f4d44812f3f84c4db2c91ba76af08a89" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 40857 } health_report { overall_health: UNKNOWN } } }
I20260809 02:05:39.353899 19126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:39.349992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41070 (local address 127.1.192.194:38117)
0809 02:05:39.350138 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0809 02:05:39.350141 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:05:39.350155 (+ 14us) server_negotiation.cc:407] Connection header received
0809 02:05:39.350314 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:05:39.350316 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:05:39.350375 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:05:39.350468 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:05:39.350873 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:39.351388 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:39.352284 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:39.352497 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:39.352637 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:05:39.352920 (+ 283us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:05:39.353356 (+ 436us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:05:39.353449 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 02:05:39.353631 (+ 182us) server_negotiation.cc:299] Negotiation successful
0809 02:05:39.353715 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 02:05:39.360313 19061 raft_consensus.cc:1261] T 679fccce20264d8aad85bf8c30572606 P f4d44812f3f84c4db2c91ba76af08a89 [term 1 FOLLOWER]: Refusing update from remote peer e8a36744066c43debf42fc9fa6d47efd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:05:39.360332 18775 raft_consensus.cc:1261] T 679fccce20264d8aad85bf8c30572606 P 1a567ef48489496e8e8a33b6a9329eda [term 1 FOLLOWER]: Refusing update from remote peer e8a36744066c43debf42fc9fa6d47efd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:05:39.361104 19135 consensus_queue.cc:1035] T 679fccce20264d8aad85bf8c30572606 P e8a36744066c43debf42fc9fa6d47efd [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a567ef48489496e8e8a33b6a9329eda" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 46211 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:05:39.361351 19140 consensus_queue.cc:1035] T 679fccce20264d8aad85bf8c30572606 P e8a36744066c43debf42fc9fa6d47efd [LEADER]: Connected to new peer: Peer: permanent_uuid: "f4d44812f3f84c4db2c91ba76af08a89" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 40857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:05:39.372244 19143 mvcc.cc:205] Tried to move back new op lower bound from 7316443706811793408 to 7316443706606972928. Current Snapshot: MvccSnapshot[applied={T|T < 7316443706811793408}]
I20260809 02:05:39.374800 19144 mvcc.cc:205] Tried to move back new op lower bound from 7316443706811793408 to 7316443706606972928. Current Snapshot: MvccSnapshot[applied={T|T < 7316443706811793408 or (T in {7316443706811793408})}]
I20260809 02:05:39.383387 18618 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41212:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:05:39.383603 18618 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41212:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:05:39.386471 18618 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9d388ae00b2c47b78dd7ca03332c3071: Sending DeleteTablet for 3 replicas of tablet 679fccce20264d8aad85bf8c30572606
I20260809 02:05:39.387356 18899 tablet_service.cc:1514] Processing DeleteTablet for tablet 679fccce20264d8aad85bf8c30572606 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:05:39 UTC) from {username='kudu', principal='kudu/127.1.192.254@KRBTEST.COM'} at 127.0.0.1:41064
I20260809 02:05:39.387472 18752 tablet_service.cc:1514] Processing DeleteTablet for tablet 679fccce20264d8aad85bf8c30572606 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:05:39 UTC) from {username='kudu', principal='kudu/127.1.192.254@KRBTEST.COM'} at 127.0.0.1:38724
I20260809 02:05:39.387573 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 18687
I20260809 02:05:39.387475 19040 tablet_service.cc:1514] Processing DeleteTablet for tablet 679fccce20264d8aad85bf8c30572606 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:05:39 UTC) from {username='kudu', principal='kudu/127.1.192.254@KRBTEST.COM'} at 127.0.0.1:45322
I20260809 02:05:39.390605 19155 tablet_replica.cc:330] T 679fccce20264d8aad85bf8c30572606 P f4d44812f3f84c4db2c91ba76af08a89: stopping tablet replica
I20260809 02:05:39.390949 19155 raft_consensus.cc:2229] T 679fccce20264d8aad85bf8c30572606 P f4d44812f3f84c4db2c91ba76af08a89 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:05:39.391304 19155 raft_consensus.cc:2258] T 679fccce20264d8aad85bf8c30572606 P f4d44812f3f84c4db2c91ba76af08a89 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:05:39.393451 19155 ts_tablet_manager.cc:1905] T 679fccce20264d8aad85bf8c30572606 P f4d44812f3f84c4db2c91ba76af08a89: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:05:39.396940 19154 tablet_replica.cc:330] T 679fccce20264d8aad85bf8c30572606 P e8a36744066c43debf42fc9fa6d47efd: stopping tablet replica
I20260809 02:05:39.397347 19154 raft_consensus.cc:2229] T 679fccce20264d8aad85bf8c30572606 P e8a36744066c43debf42fc9fa6d47efd [term 1 LEADER]: Raft consensus shutting down.
I20260809 02:05:39.397531 19155 ts_tablet_manager.cc:1918] T 679fccce20264d8aad85bf8c30572606 P f4d44812f3f84c4db2c91ba76af08a89: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:05:39.397715 19155 log.cc:1196] T 679fccce20264d8aad85bf8c30572606 P f4d44812f3f84c4db2c91ba76af08a89: Deleting WAL directory at /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-2/wal/wals/679fccce20264d8aad85bf8c30572606
I20260809 02:05:39.397912 19154 raft_consensus.cc:2258] T 679fccce20264d8aad85bf8c30572606 P e8a36744066c43debf42fc9fa6d47efd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:05:39.398167 19155 ts_tablet_manager.cc:1939] T 679fccce20264d8aad85bf8c30572606 P f4d44812f3f84c4db2c91ba76af08a89: Deleting consensus metadata
I20260809 02:05:39.399768 19154 ts_tablet_manager.cc:1905] T 679fccce20264d8aad85bf8c30572606 P e8a36744066c43debf42fc9fa6d47efd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:05:39.399861 18605 catalog_manager.cc:4841] TS f4d44812f3f84c4db2c91ba76af08a89 (127.1.192.195:40857): tablet 679fccce20264d8aad85bf8c30572606 (table test-table [id=e86faf1482d4457fba4341034dabc7ec]) successfully deleted
I20260809 02:05:39.401144 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 18832
W20260809 02:05:39.401212 18606 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.192.193:46211 (error 108)
I20260809 02:05:39.404379 19157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:39.403792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.193:46211 (local address 127.0.0.1:38730)
0809 02:05:39.404118 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:39.404226 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.193:46211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":61,"threads_started":1}
W20260809 02:05:39.404531 18606 catalog_manager.cc:4584] TS 1a567ef48489496e8e8a33b6a9329eda (127.1.192.193:46211): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 679fccce20264d8aad85bf8c30572606: Network error: Client connection negotiation failed: client connection to 127.1.192.193:46211: connect: Connection refused (error 111)
I20260809 02:05:39.410035 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 18975
I20260809 02:05:39.411811 19157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:39.411484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.194:38117 (local address 127.0.0.1:41084)
0809 02:05:39.411630 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:39.411694 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.194:38117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260809 02:05:39.418845 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 18585
I20260809 02:05:59.427110 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.192.254:42427
--webserver_interface=127.1.192.254
--webserver_port=40169
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab
--principal=kudu/127.1.192.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:33273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.192.254:42427
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:05:59.546747 19158 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:05:59.547055 19158 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:05:59.547098 19158 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:05:59.551424 19158 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 02:05:59.551505 19158 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 02:05:59.551522 19158 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 02:05:59.551543 19158 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:05:59.551564 19158 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:05:59.551584 19158 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:05:59.551649 19158 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:05:59.551672 19158 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:05:59.557523 19158 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:33273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.1.192.254:42427
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab
--principal=kudu/127.1.192.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.192.254:42427
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.192.254
--webserver_port=40169
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19158
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:05:59.558746 19158 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:05:59.559859 19158 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 02:05:59.574679 19158 server_base.cc:1034] running on GCE node
W20260809 02:05:59.574625 19166 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:05:59.574622 19164 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:05:59.574618 19168 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:05:59.575707 19158 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:05:59.576999 19158 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:05:59.578207 19158 hybrid_clock.cc:648] HybridClock initialized: now 1786241159578179 us; error 44 us; skew 500 ppm
Aug 09 02:05:59 dist-test-slave-ss4n krb5kdc[18571](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241159, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:05:59.584776 19158 init.cc:377] Logged in from keytab as kudu/127.1.192.254@KRBTEST.COM (short username kudu)
I20260809 02:05:59.585877 19158 webserver.cc:460] Webserver started at http://127.1.192.254:40169/ using document root <none> and password file <none>
I20260809 02:05:59.586510 19158 fs_manager.cc:359] Metadata directory not provided
I20260809 02:05:59.586627 19158 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:05:59.591087 19158 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 02:05:59.594040 19175 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:05:59.595441 19158 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 02:05:59.595563 19158 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/master-0/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/master-0/wal
uuid: "9d388ae00b2c47b78dd7ca03332c3071"
format_stamp: "Formatted at 2026-08-09 02:05:38 on dist-test-slave-ss4n"
server_key: "531e65c25971eaa04e81ad921adb7ed0"
server_key_iv: "e6e3375925835d2446c298332cab4ecd"
server_key_version: "encryptionkey@0"
I20260809 02:05:59.596200 19158 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:05:59.608450 19158 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:05:59.611600 19158 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:05:59.611778 19158 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:05:59.619845 19158 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.254:42427
I20260809 02:05:59.620201 19231 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.254:42427 every 8 connection(s)
I20260809 02:05:59.621079 19158 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
I20260809 02:05:59.625917 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 19158
I20260809 02:05:59.626152 19232 sys_catalog.cc:263] Verifying existing consensus state
I20260809 02:05:59.626559 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.192.193:46211
--local_ip_for_outbound_sockets=127.1.192.193
--tserver_master_addrs=127.1.192.254:42427
--webserver_port=37133
--webserver_interface=127.1.192.193
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.193.keytab
--principal=kudu/127.1.192.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:33273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 02:05:59.628125 19232 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d388ae00b2c47b78dd7ca03332c3071: Bootstrap starting.
I20260809 02:05:59.655746 19232 log.cc:824] T 00000000000000000000000000000000 P 9d388ae00b2c47b78dd7ca03332c3071: Log is configured to *not* fsync() on all Append() calls
I20260809 02:05:59.668344 19232 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d388ae00b2c47b78dd7ca03332c3071: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 02:05:59.668917 19232 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d388ae00b2c47b78dd7ca03332c3071: Bootstrap complete.
I20260809 02:05:59.675246 19232 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9d388ae00b2c47b78dd7ca03332c3071 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d388ae00b2c47b78dd7ca03332c3071" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 42427 } }
I20260809 02:05:59.675853 19232 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9d388ae00b2c47b78dd7ca03332c3071 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d388ae00b2c47b78dd7ca03332c3071, State: Initialized, Role: FOLLOWER
I20260809 02:05:59.676488 19232 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9d388ae00b2c47b78dd7ca03332c3071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d388ae00b2c47b78dd7ca03332c3071" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 42427 } }
I20260809 02:05:59.676687 19232 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9d388ae00b2c47b78dd7ca03332c3071 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:05:59.676791 19232 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9d388ae00b2c47b78dd7ca03332c3071 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:05:59.676903 19232 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9d388ae00b2c47b78dd7ca03332c3071 [term 1 FOLLOWER]: Advancing to term 2
I20260809 02:05:59.680030 19232 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9d388ae00b2c47b78dd7ca03332c3071 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d388ae00b2c47b78dd7ca03332c3071" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 42427 } }
I20260809 02:05:59.680480 19232 leader_election.cc:302] T 00000000000000000000000000000000 P 9d388ae00b2c47b78dd7ca03332c3071 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9d388ae00b2c47b78dd7ca03332c3071; no voters:
I20260809 02:05:59.680879 19232 leader_election.cc:288] T 00000000000000000000000000000000 P 9d388ae00b2c47b78dd7ca03332c3071 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 02:05:59.681021 19238 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9d388ae00b2c47b78dd7ca03332c3071 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 02:05:59.681335 19238 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9d388ae00b2c47b78dd7ca03332c3071 [term 2 LEADER]: Becoming Leader. State: Replica: 9d388ae00b2c47b78dd7ca03332c3071, State: Running, Role: LEADER
I20260809 02:05:59.681740 19238 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9d388ae00b2c47b78dd7ca03332c3071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d388ae00b2c47b78dd7ca03332c3071" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 42427 } }
I20260809 02:05:59.682442 19232 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9d388ae00b2c47b78dd7ca03332c3071 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:05:59.683218 19239 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d388ae00b2c47b78dd7ca03332c3071 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "9d388ae00b2c47b78dd7ca03332c3071" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d388ae00b2c47b78dd7ca03332c3071" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 42427 } } }
I20260809 02:05:59.683343 19239 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d388ae00b2c47b78dd7ca03332c3071 [sys.catalog]: This master's current role is: LEADER
I20260809 02:05:59.683905 19240 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d388ae00b2c47b78dd7ca03332c3071 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9d388ae00b2c47b78dd7ca03332c3071. Latest consensus state: current_term: 2 leader_uuid: "9d388ae00b2c47b78dd7ca03332c3071" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d388ae00b2c47b78dd7ca03332c3071" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 42427 } } }
I20260809 02:05:59.684027 19240 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d388ae00b2c47b78dd7ca03332c3071 [sys.catalog]: This master's current role is: LEADER
I20260809 02:05:59.684304 19245 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:05:59.689044 19245 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:05:59.689908 19245 catalog_manager.cc:1259] Loaded cluster ID: 7ae31931097d4e48b0910208ee0e9f08
I20260809 02:05:59.689977 19245 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:05:59.692493 19245 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:05:59.693241 19245 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 9d388ae00b2c47b78dd7ca03332c3071: Loaded TSK: 0
I20260809 02:05:59.705469 19245 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9d388ae00b2c47b78dd7ca03332c3071: Generated new TSK 1
I20260809 02:05:59.705809 19245 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 02:05:59.774325 19235 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:05:59.774627 19235 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:05:59.774708 19235 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:05:59.779595 19235 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:05:59.779691 19235 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:05:59.779793 19235 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.192.193
I20260809 02:05:59.785771 19235 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:33273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.193.keytab
--principal=kudu/127.1.192.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.192.193:46211
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.192.193
--webserver_port=37133
--tserver_master_addrs=127.1.192.254:42427
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19235
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.192.193
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:05:59.786919 19235 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:05:59.788009 19235 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:05:59.797940 19263 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:05:59.797951 19262 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:05:59.797940 19265 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:05:59.798434 19235 server_base.cc:1034] running on GCE node
I20260809 02:05:59.799095 19235 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:05:59.799957 19235 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:05:59.801136 19235 hybrid_clock.cc:648] HybridClock initialized: now 1786241159801090 us; error 60 us; skew 500 ppm
Aug 09 02:05:59 dist-test-slave-ss4n krb5kdc[18571](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241159, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:05:59.804349 19235 init.cc:377] Logged in from keytab as kudu/127.1.192.193@KRBTEST.COM (short username kudu)
I20260809 02:05:59.805558 19235 webserver.cc:460] Webserver started at http://127.1.192.193:37133/ using document root <none> and password file <none>
I20260809 02:05:59.806270 19235 fs_manager.cc:359] Metadata directory not provided
I20260809 02:05:59.806360 19235 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:05:59.810668 19235 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260809 02:05:59.813189 19272 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:05:59.814591 19235 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260809 02:05:59.814746 19235 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-0/wal
uuid: "1a567ef48489496e8e8a33b6a9329eda"
format_stamp: "Formatted at 2026-08-09 02:05:38 on dist-test-slave-ss4n"
server_key: "194b68000e2f13498410a1257ea231ce"
server_key_iv: "5f1024baeffe9f33765e4e4ad219ca64"
server_key_version: "encryptionkey@0"
I20260809 02:05:59.815333 19235 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:05:59.832858 19235 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:05:59.836483 19235 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:05:59.836724 19235 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:05:59.837307 19235 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:05:59.838434 19279 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 02:05:59.842612 19235 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 02:05:59.842723 19235 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.004s user 0.000s sys 0.000s
I20260809 02:05:59.842820 19235 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 02:05:59.844717 19235 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 02:05:59.844799 19235 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260809 02:05:59.845057 19279 tablet_bootstrap.cc:492] T 679fccce20264d8aad85bf8c30572606 P 1a567ef48489496e8e8a33b6a9329eda: Bootstrap starting.
I20260809 02:05:59.856808 19235 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.193:46211
I20260809 02:05:59.856910 19392 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.193:46211 every 8 connection(s)
I20260809 02:05:59.858062 19235 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-0/data/info.pb
I20260809 02:05:59.865350 19279 log.cc:824] T 679fccce20264d8aad85bf8c30572606 P 1a567ef48489496e8e8a33b6a9329eda: Log is configured to *not* fsync() on all Append() calls
I20260809 02:05:59.867328 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 19235
I20260809 02:05:59.867851 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.192.194:38117
--local_ip_for_outbound_sockets=127.1.192.194
--tserver_master_addrs=127.1.192.254:42427
--webserver_port=36429
--webserver_interface=127.1.192.194
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.194.keytab
--principal=kudu/127.1.192.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:33273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 02:05:59 dist-test-slave-ss4n krb5kdc[18571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241159, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.193@KRBTEST.COM for kudu/127.1.192.254@KRBTEST.COM
I20260809 02:05:59.873767 19279 tablet_bootstrap.cc:492] T 679fccce20264d8aad85bf8c30572606 P 1a567ef48489496e8e8a33b6a9329eda: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 02:05:59.874202 19279 tablet_bootstrap.cc:492] T 679fccce20264d8aad85bf8c30572606 P 1a567ef48489496e8e8a33b6a9329eda: Bootstrap complete.
I20260809 02:05:59.875659 19279 ts_tablet_manager.cc:1397] T 679fccce20264d8aad85bf8c30572606 P 1a567ef48489496e8e8a33b6a9329eda: Time spent bootstrapping tablet: real 0.031s user 0.014s sys 0.014s
I20260809 02:05:59.878307 19395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:59.860241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:42427 (local address 127.1.192.193:37227)
0809 02:05:59.860758 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:59.860816 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0809 02:05:59.861801 (+ 985us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:05:59.863252 (+ 1451us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:05:59.863262 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:05:59.863875 (+ 613us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:05:59.864367 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:05:59.864382 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:05:59.866213 (+ 1831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:05:59.866220 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:05:59.867308 (+ 1088us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:05:59.867320 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:05:59.868444 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:05:59.868448 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:05:59.868600 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:05:59.869274 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:05:59.869292 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:05:59.871382 (+ 2090us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:05:59.874461 (+ 3079us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:05:59.874467 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:05:59.874480 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:05:59.874927 (+ 447us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:05:59.875408 (+ 481us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:05:59.875414 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:05:59.875419 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:05:59.875596 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:05:59.876484 (+ 888us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:05:59.876490 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:05:59.876769 (+ 279us) client_negotiation.cc:769] Sending connection context
0809 02:05:59.876966 (+ 197us) client_negotiation.cc:240] Negotiation successful
0809 02:05:59.877298 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":154,"threads_started":1}
I20260809 02:05:59.878597 19396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:59.860301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.193:37227 (local address 127.1.192.254:42427)
0809 02:05:59.860838 (+ 537us) server_negotiation.cc:206] Beginning negotiation
0809 02:05:59.860856 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0809 02:05:59.860909 (+ 53us) server_negotiation.cc:407] Connection header received
0809 02:05:59.862256 (+ 1347us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:05:59.862271 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:05:59.862673 (+ 402us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:05:59.863112 (+ 439us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:05:59.864554 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:59.866048 (+ 1494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:59.867456 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:59.868105 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:59.868268 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:05:59.871622 (+ 3354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:05:59.871632 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:05:59.871652 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:05:59.871695 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:05:59.874314 (+ 2619us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:05:59.875077 (+ 763us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:05:59.875087 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:05:59.875096 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:05:59.875287 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:05:59.875725 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:05:59.875732 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:05:59.875736 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:05:59.876337 (+ 601us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:05:59.876522 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0809 02:05:59.877151 (+ 629us) server_negotiation.cc:299] Negotiation successful
0809 02:05:59.877571 (+ 420us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":395,"thread_start_us":144,"threads_started":1}
I20260809 02:05:59.878495 19279 raft_consensus.cc:348] T 679fccce20264d8aad85bf8c30572606 P 1a567ef48489496e8e8a33b6a9329eda [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a36744066c43debf42fc9fa6d47efd" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38117 } } peers { permanent_uuid: "1a567ef48489496e8e8a33b6a9329eda" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 46211 } } peers { permanent_uuid: "f4d44812f3f84c4db2c91ba76af08a89" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 40857 } }
I20260809 02:05:59.879024 19279 raft_consensus.cc:729] T 679fccce20264d8aad85bf8c30572606 P 1a567ef48489496e8e8a33b6a9329eda [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a567ef48489496e8e8a33b6a9329eda, State: Initialized, Role: FOLLOWER
I20260809 02:05:59.879516 19279 consensus_queue.cc:260] T 679fccce20264d8aad85bf8c30572606 P 1a567ef48489496e8e8a33b6a9329eda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a36744066c43debf42fc9fa6d47efd" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38117 } } peers { permanent_uuid: "1a567ef48489496e8e8a33b6a9329eda" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 46211 } } peers { permanent_uuid: "f4d44812f3f84c4db2c91ba76af08a89" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 40857 } }
I20260809 02:05:59.880559 19279 ts_tablet_manager.cc:1428] T 679fccce20264d8aad85bf8c30572606 P 1a567ef48489496e8e8a33b6a9329eda: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
I20260809 02:05:59.881558 19393 heartbeater.cc:347] Connected to a master server at 127.1.192.254:42427
I20260809 02:05:59.881834 19393 heartbeater.cc:464] Registering TS with master...
I20260809 02:05:59.882551 19393 heartbeater.cc:511] Master 127.1.192.254:42427 requested a full tablet report, sending...
I20260809 02:05:59.885283 19192 ts_manager.cc:194] Registered new tserver with Master: 1a567ef48489496e8e8a33b6a9329eda (127.1.192.193:46211)
I20260809 02:05:59.889089 19192 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.192.193@KRBTEST.COM'} at 127.1.192.193:37227
I20260809 02:05:59.892092 19393 heartbeater.cc:503] Master 127.1.192.254:42427 was elected leader, sending a full tablet report...
Aug 09 02:05:59 dist-test-slave-ss4n krb5kdc[18571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241159, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.254@KRBTEST.COM for kudu/127.1.192.193@KRBTEST.COM
I20260809 02:05:59.900135 19400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:59.888195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.193:46211 (local address 127.0.0.1:53506)
0809 02:05:59.888652 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0809 02:05:59.888700 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 02:05:59.888828 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:05:59.889631 (+ 803us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:05:59.889639 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:05:59.890130 (+ 491us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:05:59.890278 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:05:59.890287 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:05:59.891755 (+ 1468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:05:59.891759 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:05:59.892890 (+ 1131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:05:59.892902 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:05:59.893755 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:05:59.893760 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:05:59.893891 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:05:59.894538 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:05:59.894557 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:05:59.896185 (+ 1628us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:05:59.898256 (+ 2071us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:05:59.898268 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:05:59.898274 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:05:59.898532 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:05:59.898863 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:05:59.898866 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:05:59.898868 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:05:59.898955 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:05:59.899436 (+ 481us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:05:59.899442 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:05:59.899571 (+ 129us) client_negotiation.cc:769] Sending connection context
0809 02:05:59.899723 (+ 152us) client_negotiation.cc:240] Negotiation successful
0809 02:05:59.899935 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":140,"threads_started":1}
I20260809 02:05:59.900153 19401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:05:59.888265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53506 (local address 127.1.192.193:46211)
0809 02:05:59.889106 (+ 841us) server_negotiation.cc:206] Beginning negotiation
0809 02:05:59.889120 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 02:05:59.889143 (+ 23us) server_negotiation.cc:407] Connection header received
0809 02:05:59.889216 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:05:59.889221 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:05:59.889421 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:05:59.889590 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:05:59.890413 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:59.891606 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:59.893010 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:05:59.893617 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:05:59.893746 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:05:59.896333 (+ 2587us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:05:59.896342 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:05:59.896356 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:05:59.896388 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:05:59.898134 (+ 1746us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:05:59.898663 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:05:59.898670 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:05:59.898672 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:05:59.898758 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:05:59.899048 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:05:59.899053 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:05:59.899055 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:05:59.899309 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:05:59.899444 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0809 02:05:59.899817 (+ 373us) server_negotiation.cc:299] Negotiation successful
0809 02:05:59.899985 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":743,"thread_start_us":115,"threads_started":1}
I20260809 02:05:59.901705 19322 tablet_service.cc:1514] Processing DeleteTablet for tablet 679fccce20264d8aad85bf8c30572606 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:05:39 UTC) from {username='kudu', principal='kudu/127.1.192.254@KRBTEST.COM'} at 127.0.0.1:53506
I20260809 02:05:59.902832 19402 tablet_replica.cc:330] stopping tablet replica
I20260809 02:05:59.903131 19402 raft_consensus.cc:2229] T 679fccce20264d8aad85bf8c30572606 P 1a567ef48489496e8e8a33b6a9329eda [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:05:59.903385 19402 raft_consensus.cc:2258] T 679fccce20264d8aad85bf8c30572606 P 1a567ef48489496e8e8a33b6a9329eda [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:05:59.904413 19402 ts_tablet_manager.cc:1905] T 679fccce20264d8aad85bf8c30572606 P 1a567ef48489496e8e8a33b6a9329eda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:05:59.908013 19402 ts_tablet_manager.cc:1918] T 679fccce20264d8aad85bf8c30572606 P 1a567ef48489496e8e8a33b6a9329eda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:05:59.908133 19402 log.cc:1196] T 679fccce20264d8aad85bf8c30572606 P 1a567ef48489496e8e8a33b6a9329eda: Deleting WAL directory at /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-0/wal/wals/679fccce20264d8aad85bf8c30572606
I20260809 02:05:59.908411 19402 ts_tablet_manager.cc:1939] T 679fccce20264d8aad85bf8c30572606 P 1a567ef48489496e8e8a33b6a9329eda: Deleting consensus metadata
I20260809 02:05:59.910426 19178 catalog_manager.cc:4841] TS 1a567ef48489496e8e8a33b6a9329eda (127.1.192.193:46211): tablet 679fccce20264d8aad85bf8c30572606 (table test-table [id=e86faf1482d4457fba4341034dabc7ec]) successfully deleted
W20260809 02:06:00.001842 19398 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:06:00.002216 19398 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:06:00.002313 19398 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:06:00.006994 19398 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:06:00.007061 19398 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:06:00.007125 19398 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.192.194
I20260809 02:06:00.012573 19398 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:33273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.194.keytab
--principal=kudu/127.1.192.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.192.194:38117
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.192.194
--webserver_port=36429
--tserver_master_addrs=127.1.192.254:42427
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19398
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.192.194
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:06:00.013764 19398 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:06:00.014853 19398 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:06:00.024925 19407 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:06:00.024930 19410 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:06:00.024978 19408 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:06:00.025154 19398 server_base.cc:1034] running on GCE node
I20260809 02:06:00.025770 19398 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:06:00.026453 19398 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:06:00.027616 19398 hybrid_clock.cc:648] HybridClock initialized: now 1786241160027597 us; error 32 us; skew 500 ppm
Aug 09 02:06:00 dist-test-slave-ss4n krb5kdc[18571](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241160, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:06:00.030522 19398 init.cc:377] Logged in from keytab as kudu/127.1.192.194@KRBTEST.COM (short username kudu)
I20260809 02:06:00.031450 19398 webserver.cc:460] Webserver started at http://127.1.192.194:36429/ using document root <none> and password file <none>
I20260809 02:06:00.032044 19398 fs_manager.cc:359] Metadata directory not provided
I20260809 02:06:00.032155 19398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:06:00.036441 19398 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260809 02:06:00.038802 19417 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:06:00.040100 19398 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 02:06:00.040267 19398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-1/wal
uuid: "e8a36744066c43debf42fc9fa6d47efd"
format_stamp: "Formatted at 2026-08-09 02:05:38 on dist-test-slave-ss4n"
server_key: "8214780cf82e86dfdcc54772a3e6d20a"
server_key_iv: "c4fe70c99d60c3a0f7122d7260b3a121"
server_key_version: "encryptionkey@0"
I20260809 02:06:00.040833 19398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:06:00.056229 19398 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:06:00.060141 19398 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:06:00.060384 19398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:06:00.061129 19398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:06:00.062563 19424 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 02:06:00.067365 19398 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 02:06:00.067538 19398 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20260809 02:06:00.067633 19398 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 02:06:00.069468 19398 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 02:06:00.069614 19398 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.002s
I20260809 02:06:00.069904 19424 tablet_bootstrap.cc:492] T 679fccce20264d8aad85bf8c30572606 P e8a36744066c43debf42fc9fa6d47efd: Bootstrap starting.
I20260809 02:06:00.076534 19424 log.cc:824] T 679fccce20264d8aad85bf8c30572606 P e8a36744066c43debf42fc9fa6d47efd: Log is configured to *not* fsync() on all Append() calls
I20260809 02:06:00.080827 19398 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.194:38117
I20260809 02:06:00.080911 19538 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.194:38117 every 8 connection(s)
I20260809 02:06:00.081820 19398 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-1/data/info.pb
I20260809 02:06:00.086604 19424 tablet_bootstrap.cc:492] T 679fccce20264d8aad85bf8c30572606 P e8a36744066c43debf42fc9fa6d47efd: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 02:06:00.087066 19424 tablet_bootstrap.cc:492] T 679fccce20264d8aad85bf8c30572606 P e8a36744066c43debf42fc9fa6d47efd: Bootstrap complete.
I20260809 02:06:00.087401 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 19398
I20260809 02:06:00.087522 19424 ts_tablet_manager.cc:1397] T 679fccce20264d8aad85bf8c30572606 P e8a36744066c43debf42fc9fa6d47efd: Time spent bootstrapping tablet: real 0.018s user 0.005s sys 0.010s
I20260809 02:06:00.087967 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.192.195:40857
--local_ip_for_outbound_sockets=127.1.192.195
--tserver_master_addrs=127.1.192.254:42427
--webserver_port=41081
--webserver_interface=127.1.192.195
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.195.keytab
--principal=kudu/127.1.192.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:33273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 02:06:00.091419 19424 raft_consensus.cc:348] T 679fccce20264d8aad85bf8c30572606 P e8a36744066c43debf42fc9fa6d47efd [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a36744066c43debf42fc9fa6d47efd" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38117 } } peers { permanent_uuid: "1a567ef48489496e8e8a33b6a9329eda" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 46211 } } peers { permanent_uuid: "f4d44812f3f84c4db2c91ba76af08a89" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 40857 } }
I20260809 02:06:00.092125 19424 raft_consensus.cc:729] T 679fccce20264d8aad85bf8c30572606 P e8a36744066c43debf42fc9fa6d47efd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8a36744066c43debf42fc9fa6d47efd, State: Initialized, Role: FOLLOWER
Aug 09 02:06:00 dist-test-slave-ss4n krb5kdc[18571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241160, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.194@KRBTEST.COM for kudu/127.1.192.254@KRBTEST.COM
I20260809 02:06:00.092762 19424 consensus_queue.cc:260] T 679fccce20264d8aad85bf8c30572606 P e8a36744066c43debf42fc9fa6d47efd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8a36744066c43debf42fc9fa6d47efd" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38117 } } peers { permanent_uuid: "1a567ef48489496e8e8a33b6a9329eda" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 46211 } } peers { permanent_uuid: "f4d44812f3f84c4db2c91ba76af08a89" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 40857 } }
I20260809 02:06:00.094189 19424 ts_tablet_manager.cc:1428] T 679fccce20264d8aad85bf8c30572606 P e8a36744066c43debf42fc9fa6d47efd: Time spent starting tablet: real 0.007s user 0.007s sys 0.001s
I20260809 02:06:00.097844 19396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:00.083819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.194:55333 (local address 127.1.192.254:42427)
0809 02:06:00.083968 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0809 02:06:00.083971 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:06:00.084882 (+ 911us) server_negotiation.cc:407] Connection header received
0809 02:06:00.085895 (+ 1013us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:06:00.085898 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:06:00.085964 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:06:00.086068 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:06:00.087323 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:00.087957 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:00.089251 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:00.089563 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:00.089701 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:00.093770 (+ 4069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:06:00.093774 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:06:00.093776 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:06:00.093805 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:06:00.095484 (+ 1679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:06:00.096093 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:06:00.096097 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:06:00.096098 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:06:00.096207 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:06:00.096596 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:06:00.096599 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:06:00.096601 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:06:00.096806 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:06:00.096919 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 02:06:00.097456 (+ 537us) server_negotiation.cc:299] Negotiation successful
0809 02:06:00.097657 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 02:06:00.098701 19541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:00.084000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:42427 (local address 127.1.192.194:55333)
0809 02:06:00.084725 (+ 725us) negotiation.cc:107] Waiting for socket to connect
0809 02:06:00.084769 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 02:06:00.085688 (+ 919us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:06:00.086222 (+ 534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:06:00.086228 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:06:00.086567 (+ 339us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:06:00.087105 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:06:00.087120 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:06:00.088273 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:06:00.088276 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:06:00.089121 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:06:00.089131 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:06:00.089703 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:06:00.089707 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:06:00.089959 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:00.091407 (+ 1448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:06:00.091430 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:06:00.093542 (+ 2112us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:06:00.095634 (+ 2092us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:06:00.095641 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:06:00.095657 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:06:00.095941 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:06:00.096315 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:06:00.096318 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:06:00.096320 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:06:00.096476 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:06:00.096918 (+ 442us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:06:00.096925 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:06:00.097206 (+ 281us) client_negotiation.cc:769] Sending connection context
0809 02:06:00.097407 (+ 201us) client_negotiation.cc:240] Negotiation successful
0809 02:06:00.097801 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":163,"threads_started":1}
I20260809 02:06:00.100003 19539 heartbeater.cc:347] Connected to a master server at 127.1.192.254:42427
I20260809 02:06:00.100312 19539 heartbeater.cc:464] Registering TS with master...
I20260809 02:06:00.100893 19539 heartbeater.cc:511] Master 127.1.192.254:42427 requested a full tablet report, sending...
I20260809 02:06:00.102860 19192 ts_manager.cc:194] Registered new tserver with Master: e8a36744066c43debf42fc9fa6d47efd (127.1.192.194:38117)
I20260809 02:06:00.104110 19192 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.192.194@KRBTEST.COM'} at 127.1.192.194:55333
I20260809 02:06:00.106449 19539 heartbeater.cc:503] Master 127.1.192.254:42427 was elected leader, sending a full tablet report...
Aug 09 02:06:00 dist-test-slave-ss4n krb5kdc[18571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241159, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.254@KRBTEST.COM for kudu/127.1.192.194@KRBTEST.COM
I20260809 02:06:00.114990 19546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:00.103926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57140 (local address 127.1.192.194:38117)
0809 02:06:00.104475 (+ 549us) server_negotiation.cc:206] Beginning negotiation
0809 02:06:00.104488 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 02:06:00.104512 (+ 24us) server_negotiation.cc:407] Connection header received
0809 02:06:00.104587 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:06:00.104593 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:06:00.104844 (+ 251us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:06:00.104992 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:06:00.105444 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:00.106827 (+ 1383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:00.107702 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:00.108340 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:00.108462 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:00.111087 (+ 2625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:06:00.111095 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:06:00.111107 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:06:00.111131 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:06:00.112888 (+ 1757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:06:00.113429 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:06:00.113433 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:06:00.113435 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:06:00.113582 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:06:00.113924 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:06:00.113927 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:06:00.113928 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:06:00.114255 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:06:00.114363 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 02:06:00.114718 (+ 355us) server_negotiation.cc:299] Negotiation successful
0809 02:06:00.114858 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":441,"thread_start_us":129,"threads_started":1}
I20260809 02:06:00.114979 19400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:00.103798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.194:38117 (local address 127.0.0.1:57140)
0809 02:06:00.103995 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0809 02:06:00.104013 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 02:06:00.104114 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:06:00.104997 (+ 883us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:06:00.105001 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:06:00.105209 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:06:00.105313 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:06:00.105321 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:06:00.106999 (+ 1678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:06:00.107004 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:06:00.107577 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:06:00.107586 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:06:00.108461 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:06:00.108465 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:06:00.108566 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:00.109151 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:06:00.109167 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:06:00.110939 (+ 1772us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:06:00.113004 (+ 2065us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:06:00.113008 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:06:00.113011 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:06:00.113304 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:06:00.113681 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:06:00.113684 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:06:00.113689 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:06:00.113795 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:06:00.114363 (+ 568us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:06:00.114366 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:06:00.114480 (+ 114us) client_negotiation.cc:769] Sending connection context
0809 02:06:00.114607 (+ 127us) client_negotiation.cc:240] Negotiation successful
0809 02:06:00.114772 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118}
I20260809 02:06:00.116221 19468 tablet_service.cc:1514] Processing DeleteTablet for tablet 679fccce20264d8aad85bf8c30572606 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:05:39 UTC) from {username='kudu', principal='kudu/127.1.192.254@KRBTEST.COM'} at 127.0.0.1:57140
I20260809 02:06:00.117138 19547 tablet_replica.cc:330] stopping tablet replica
I20260809 02:06:00.117446 19547 raft_consensus.cc:2229] T 679fccce20264d8aad85bf8c30572606 P e8a36744066c43debf42fc9fa6d47efd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:06:00.117750 19547 raft_consensus.cc:2258] T 679fccce20264d8aad85bf8c30572606 P e8a36744066c43debf42fc9fa6d47efd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:06:00.118815 19547 ts_tablet_manager.cc:1905] T 679fccce20264d8aad85bf8c30572606 P e8a36744066c43debf42fc9fa6d47efd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:06:00.122622 19547 ts_tablet_manager.cc:1918] T 679fccce20264d8aad85bf8c30572606 P e8a36744066c43debf42fc9fa6d47efd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:06:00.122712 19547 log.cc:1196] T 679fccce20264d8aad85bf8c30572606 P e8a36744066c43debf42fc9fa6d47efd: Deleting WAL directory at /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-1/wal/wals/679fccce20264d8aad85bf8c30572606
I20260809 02:06:00.122991 19547 ts_tablet_manager.cc:1939] T 679fccce20264d8aad85bf8c30572606 P e8a36744066c43debf42fc9fa6d47efd: Deleting consensus metadata
I20260809 02:06:00.127739 19177 catalog_manager.cc:4841] TS e8a36744066c43debf42fc9fa6d47efd (127.1.192.194:38117): tablet 679fccce20264d8aad85bf8c30572606 (table test-table [id=e86faf1482d4457fba4341034dabc7ec]) successfully deleted
W20260809 02:06:00.243503 19543 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:06:00.243842 19543 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:06:00.243893 19543 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:06:00.248449 19543 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:06:00.248533 19543 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:06:00.248603 19543 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.192.195
I20260809 02:06:00.254133 19543 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:33273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.195.keytab
--principal=kudu/127.1.192.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.192.195:40857
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.192.195
--webserver_port=41081
--tserver_master_addrs=127.1.192.254:42427
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19543
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.192.195
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:06:00.255191 19543 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:06:00.256165 19543 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:06:00.266002 19553 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:06:00.266038 19555 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:06:00.266008 19552 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:06:00.266388 19543 server_base.cc:1034] running on GCE node
I20260809 02:06:00.266927 19543 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:06:00.267604 19543 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:06:00.268836 19543 hybrid_clock.cc:648] HybridClock initialized: now 1786241160268824 us; error 40 us; skew 500 ppm
Aug 09 02:06:00 dist-test-slave-ss4n krb5kdc[18571](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241160, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:06:00.271831 19543 init.cc:377] Logged in from keytab as kudu/127.1.192.195@KRBTEST.COM (short username kudu)
I20260809 02:06:00.272794 19543 webserver.cc:460] Webserver started at http://127.1.192.195:41081/ using document root <none> and password file <none>
I20260809 02:06:00.273367 19543 fs_manager.cc:359] Metadata directory not provided
I20260809 02:06:00.273530 19543 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:06:00.277436 19543 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.002s
I20260809 02:06:00.279717 19562 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:06:00.281026 19543 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 02:06:00.281172 19543 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-2/wal
uuid: "f4d44812f3f84c4db2c91ba76af08a89"
format_stamp: "Formatted at 2026-08-09 02:05:38 on dist-test-slave-ss4n"
server_key: "9bdb0b28e7ab8d3776b4a2c64ecaba1d"
server_key_iv: "6f139f9db5bf58e0ea8fd14d29fb5ccd"
server_key_version: "encryptionkey@0"
I20260809 02:06:00.281840 19543 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:06:00.291775 19543 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:06:00.295033 19543 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:06:00.295258 19543 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:06:00.295885 19543 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:06:00.296746 19543 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:06:00.296819 19543 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:06:00.296892 19543 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:06:00.296936 19543 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:06:00.306867 19543 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.195:40857
I20260809 02:06:00.306931 19686 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.195:40857 every 8 connection(s)
I20260809 02:06:00.308043 19543 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-2/data/info.pb
I20260809 02:06:00.318260 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 19543
Aug 09 02:06:00 dist-test-slave-ss4n krb5kdc[18571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241160, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.195@KRBTEST.COM for kudu/127.1.192.254@KRBTEST.COM
I20260809 02:06:00.324747 19396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:00.310207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.195:48351 (local address 127.1.192.254:42427)
0809 02:06:00.310370 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0809 02:06:00.310378 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 02:06:00.311719 (+ 1341us) server_negotiation.cc:407] Connection header received
0809 02:06:00.312783 (+ 1064us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:06:00.312787 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:06:00.312856 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:06:00.312976 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:06:00.314379 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:00.314959 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:00.316249 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:00.316509 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:00.316640 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:00.319820 (+ 3180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:06:00.319824 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:06:00.319826 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:06:00.319847 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:06:00.321634 (+ 1787us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:06:00.322659 (+ 1025us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:06:00.322662 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:06:00.322664 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:06:00.322754 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:06:00.323213 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:06:00.323216 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:06:00.323218 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:06:00.323446 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:06:00.323550 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0809 02:06:00.324456 (+ 906us) server_negotiation.cc:299] Negotiation successful
0809 02:06:00.324607 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 02:06:00.326122 19689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:00.310503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:42427 (local address 127.1.192.195:48351)
0809 02:06:00.311519 (+ 1016us) negotiation.cc:107] Waiting for socket to connect
0809 02:06:00.311569 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 02:06:00.312509 (+ 940us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:06:00.313185 (+ 676us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:06:00.313192 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:06:00.313639 (+ 447us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:06:00.314198 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:06:00.314214 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:06:00.315107 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:06:00.315112 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:06:00.316099 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:06:00.316108 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:06:00.316692 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:06:00.316697 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:06:00.316891 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:00.317690 (+ 799us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:06:00.317710 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:06:00.319644 (+ 1934us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:06:00.321909 (+ 2265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:06:00.321917 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:06:00.321939 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:06:00.322419 (+ 480us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:06:00.322857 (+ 438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:06:00.322861 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:06:00.322863 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:06:00.323098 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:06:00.323675 (+ 577us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:06:00.323683 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:06:00.324103 (+ 420us) client_negotiation.cc:769] Sending connection context
0809 02:06:00.324385 (+ 282us) client_negotiation.cc:240] Negotiation successful
0809 02:06:00.324831 (+ 446us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":767,"thread_start_us":582,"threads_started":1}
I20260809 02:06:00.327533 19687 heartbeater.cc:347] Connected to a master server at 127.1.192.254:42427
I20260809 02:06:00.327804 19687 heartbeater.cc:464] Registering TS with master...
I20260809 02:06:00.328329 19687 heartbeater.cc:511] Master 127.1.192.254:42427 requested a full tablet report, sending...
I20260809 02:06:00.329460 19192 ts_manager.cc:194] Registered new tserver with Master: f4d44812f3f84c4db2c91ba76af08a89 (127.1.192.195:40857)
I20260809 02:06:00.330175 19692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:00.319070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52128 (local address 127.1.192.254:42427)
0809 02:06:00.319471 (+ 401us) server_negotiation.cc:206] Beginning negotiation
0809 02:06:00.319476 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:06:00.319552 (+ 76us) server_negotiation.cc:407] Connection header received
0809 02:06:00.319603 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:06:00.319606 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:06:00.319699 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:06:00.319808 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:06:00.320808 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:00.321697 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:00.323152 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:00.323521 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:00.323635 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:00.325558 (+ 1923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:06:00.325563 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:06:00.325568 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:06:00.325595 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:06:00.327954 (+ 2359us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:06:00.328447 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:06:00.328453 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:06:00.328455 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:06:00.328554 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:06:00.328868 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:06:00.328873 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:06:00.328875 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:06:00.329123 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:06:00.329354 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0809 02:06:00.329934 (+ 580us) server_negotiation.cc:299] Negotiation successful
0809 02:06:00.330050 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":94,"threads_started":1}
I20260809 02:06:00.330333 19192 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.192.195@KRBTEST.COM'} at 127.1.192.195:48351
I20260809 02:06:00.332192 1795 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 02:06:00.336424 19692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:00.332880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52144 (local address 127.1.192.254:42427)
0809 02:06:00.333010 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0809 02:06:00.333014 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:06:00.333237 (+ 223us) server_negotiation.cc:407] Connection header received
0809 02:06:00.333365 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:06:00.333367 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:06:00.333452 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:06:00.333592 (+ 140us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:06:00.333913 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:00.334406 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:00.335307 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:00.335561 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:00.335657 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:00.335940 (+ 283us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:06:00.336070 (+ 130us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0809 02:06:00.336278 (+ 208us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:52144: token expired
Metrics: {"server-negotiator.queue_time_us":51}
W20260809 02:06:00.336481 19692 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:52144: token expired
W20260809 02:06:00.336848 19118 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.1.192.254:42427)
I20260809 02:06:00.343792 19692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:00.337268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52156 (local address 127.1.192.254:42427)
0809 02:06:00.337373 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0809 02:06:00.337377 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:06:00.337396 (+ 19us) server_negotiation.cc:407] Connection header received
0809 02:06:00.337444 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:06:00.337446 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:06:00.337520 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:06:00.337599 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:06:00.338196 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:00.338652 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:00.339391 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:00.339587 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:00.339671 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:00.340946 (+ 1275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:06:00.340948 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:06:00.340950 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:06:00.340971 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:06:00.342369 (+ 1398us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:06:00.342774 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:06:00.342776 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:06:00.342777 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:06:00.342847 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:06:00.343066 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:06:00.343068 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:06:00.343069 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:06:00.343247 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:06:00.343345 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0809 02:06:00.343574 (+ 229us) server_negotiation.cc:299] Negotiation successful
0809 02:06:00.343672 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 02:06:00.346900 19192 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52156:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 02:06:00.350250 19192 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:06:00.358101 19468 tablet_service.cc:1467] Processing CreateTablet for tablet da13c8d014e1480b93127bf1a2f3924f (DEFAULT_TABLE table=test-table [id=2a1788194d94427ea50afccde17fcf59]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:06:00.358101 19322 tablet_service.cc:1467] Processing CreateTablet for tablet da13c8d014e1480b93127bf1a2f3924f (DEFAULT_TABLE table=test-table [id=2a1788194d94427ea50afccde17fcf59]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:06:00.358767 19468 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet da13c8d014e1480b93127bf1a2f3924f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:06:00.358767 19322 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet da13c8d014e1480b93127bf1a2f3924f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:06:00.361791 19279 tablet_bootstrap.cc:492] T da13c8d014e1480b93127bf1a2f3924f P 1a567ef48489496e8e8a33b6a9329eda: Bootstrap starting.
I20260809 02:06:00.362398 19400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:00.357244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.195:40857 (local address 127.0.0.1:57836)
0809 02:06:00.357364 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0809 02:06:00.357382 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 02:06:00.357471 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:06:00.358553 (+ 1082us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:06:00.358556 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:06:00.358574 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:06:00.358695 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:06:00.358704 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:06:00.359810 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:06:00.359817 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:06:00.360931 (+ 1114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:06:00.360941 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:06:00.361937 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:06:00.361940 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:06:00.362033 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:00.362093 (+ 60us) client_negotiation.cc:769] Sending connection context
0809 02:06:00.362151 (+ 58us) client_negotiation.cc:240] Negotiation successful
0809 02:06:00.362255 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260809 02:06:00.362723 19424 tablet_bootstrap.cc:492] T da13c8d014e1480b93127bf1a2f3924f P e8a36744066c43debf42fc9fa6d47efd: Bootstrap starting.
I20260809 02:06:00.362762 19279 tablet_bootstrap.cc:654] T da13c8d014e1480b93127bf1a2f3924f P 1a567ef48489496e8e8a33b6a9329eda: Neither blocks nor log segments found. Creating new log.
I20260809 02:06:00.363387 19697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:00.357722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57836 (local address 127.1.192.195:40857)
0809 02:06:00.358069 (+ 347us) server_negotiation.cc:206] Beginning negotiation
0809 02:06:00.358082 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 02:06:00.358106 (+ 24us) server_negotiation.cc:407] Connection header received
0809 02:06:00.358182 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:06:00.358186 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:06:00.358406 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:06:00.358583 (+ 177us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:06:00.358801 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:00.359692 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:00.361068 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:00.361789 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:00.362599 (+ 810us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:00.362823 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0809 02:06:00.363072 (+ 249us) server_negotiation.cc:299] Negotiation successful
0809 02:06:00.363223 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":246,"thread_start_us":151,"threads_started":1}
I20260809 02:06:00.363729 19424 tablet_bootstrap.cc:654] T da13c8d014e1480b93127bf1a2f3924f P e8a36744066c43debf42fc9fa6d47efd: Neither blocks nor log segments found. Creating new log.
I20260809 02:06:00.364455 19279 tablet_bootstrap.cc:492] T da13c8d014e1480b93127bf1a2f3924f P 1a567ef48489496e8e8a33b6a9329eda: No bootstrap required, opened a new log
I20260809 02:06:00.364545 19279 ts_tablet_manager.cc:1397] T da13c8d014e1480b93127bf1a2f3924f P 1a567ef48489496e8e8a33b6a9329eda: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260809 02:06:00.365021 19279 raft_consensus.cc:348] T da13c8d014e1480b93127bf1a2f3924f P 1a567ef48489496e8e8a33b6a9329eda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4d44812f3f84c4db2c91ba76af08a89" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 40857 } } peers { permanent_uuid: "1a567ef48489496e8e8a33b6a9329eda" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 46211 } } peers { permanent_uuid: "e8a36744066c43debf42fc9fa6d47efd" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38117 } }
I20260809 02:06:00.365173 19279 raft_consensus.cc:374] T da13c8d014e1480b93127bf1a2f3924f P 1a567ef48489496e8e8a33b6a9329eda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:06:00.365217 19279 raft_consensus.cc:729] T da13c8d014e1480b93127bf1a2f3924f P 1a567ef48489496e8e8a33b6a9329eda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a567ef48489496e8e8a33b6a9329eda, State: Initialized, Role: FOLLOWER
I20260809 02:06:00.365350 19279 consensus_queue.cc:260] T da13c8d014e1480b93127bf1a2f3924f P 1a567ef48489496e8e8a33b6a9329eda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4d44812f3f84c4db2c91ba76af08a89" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 40857 } } peers { permanent_uuid: "1a567ef48489496e8e8a33b6a9329eda" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 46211 } } peers { permanent_uuid: "e8a36744066c43debf42fc9fa6d47efd" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38117 } }
I20260809 02:06:00.365417 19424 tablet_bootstrap.cc:492] T da13c8d014e1480b93127bf1a2f3924f P e8a36744066c43debf42fc9fa6d47efd: No bootstrap required, opened a new log
I20260809 02:06:00.365520 19424 ts_tablet_manager.cc:1397] T da13c8d014e1480b93127bf1a2f3924f P e8a36744066c43debf42fc9fa6d47efd: Time spent bootstrapping tablet: real 0.003s user 0.003s sys 0.000s
I20260809 02:06:00.365757 19279 ts_tablet_manager.cc:1428] T da13c8d014e1480b93127bf1a2f3924f P 1a567ef48489496e8e8a33b6a9329eda: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20260809 02:06:00.365983 19424 raft_consensus.cc:348] T da13c8d014e1480b93127bf1a2f3924f P e8a36744066c43debf42fc9fa6d47efd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4d44812f3f84c4db2c91ba76af08a89" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 40857 } } peers { permanent_uuid: "1a567ef48489496e8e8a33b6a9329eda" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 46211 } } peers { permanent_uuid: "e8a36744066c43debf42fc9fa6d47efd" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38117 } }
I20260809 02:06:00.366122 19424 raft_consensus.cc:374] T da13c8d014e1480b93127bf1a2f3924f P e8a36744066c43debf42fc9fa6d47efd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:06:00.366180 19424 raft_consensus.cc:729] T da13c8d014e1480b93127bf1a2f3924f P e8a36744066c43debf42fc9fa6d47efd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8a36744066c43debf42fc9fa6d47efd, State: Initialized, Role: FOLLOWER
I20260809 02:06:00.366323 19424 consensus_queue.cc:260] T da13c8d014e1480b93127bf1a2f3924f P e8a36744066c43debf42fc9fa6d47efd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4d44812f3f84c4db2c91ba76af08a89" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 40857 } } peers { permanent_uuid: "1a567ef48489496e8e8a33b6a9329eda" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 46211 } } peers { permanent_uuid: "e8a36744066c43debf42fc9fa6d47efd" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38117 } }
I20260809 02:06:00.366601 19424 ts_tablet_manager.cc:1428] T da13c8d014e1480b93127bf1a2f3924f P e8a36744066c43debf42fc9fa6d47efd: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 02:06:00.367156 19614 tablet_service.cc:1467] Processing CreateTablet for tablet da13c8d014e1480b93127bf1a2f3924f (DEFAULT_TABLE table=test-table [id=2a1788194d94427ea50afccde17fcf59]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:06:00.368443 19614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet da13c8d014e1480b93127bf1a2f3924f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:06:00.375911 19700 tablet_bootstrap.cc:492] T da13c8d014e1480b93127bf1a2f3924f P f4d44812f3f84c4db2c91ba76af08a89: Bootstrap starting.
I20260809 02:06:00.378727 19700 tablet_bootstrap.cc:654] T da13c8d014e1480b93127bf1a2f3924f P f4d44812f3f84c4db2c91ba76af08a89: Neither blocks nor log segments found. Creating new log.
I20260809 02:06:00.379518 19700 log.cc:824] T da13c8d014e1480b93127bf1a2f3924f P f4d44812f3f84c4db2c91ba76af08a89: Log is configured to *not* fsync() on all Append() calls
I20260809 02:06:00.381449 19700 tablet_bootstrap.cc:492] T da13c8d014e1480b93127bf1a2f3924f P f4d44812f3f84c4db2c91ba76af08a89: No bootstrap required, opened a new log
I20260809 02:06:00.381714 19700 ts_tablet_manager.cc:1397] T da13c8d014e1480b93127bf1a2f3924f P f4d44812f3f84c4db2c91ba76af08a89: Time spent bootstrapping tablet: real 0.006s user 0.006s sys 0.000s
I20260809 02:06:00.385232 19700 raft_consensus.cc:348] T da13c8d014e1480b93127bf1a2f3924f P f4d44812f3f84c4db2c91ba76af08a89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4d44812f3f84c4db2c91ba76af08a89" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 40857 } } peers { permanent_uuid: "1a567ef48489496e8e8a33b6a9329eda" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 46211 } } peers { permanent_uuid: "e8a36744066c43debf42fc9fa6d47efd" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38117 } }
I20260809 02:06:00.385557 19700 raft_consensus.cc:374] T da13c8d014e1480b93127bf1a2f3924f P f4d44812f3f84c4db2c91ba76af08a89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:06:00.385644 19700 raft_consensus.cc:729] T da13c8d014e1480b93127bf1a2f3924f P f4d44812f3f84c4db2c91ba76af08a89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4d44812f3f84c4db2c91ba76af08a89, State: Initialized, Role: FOLLOWER
I20260809 02:06:00.386145 19700 consensus_queue.cc:260] T da13c8d014e1480b93127bf1a2f3924f P f4d44812f3f84c4db2c91ba76af08a89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4d44812f3f84c4db2c91ba76af08a89" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 40857 } } peers { permanent_uuid: "1a567ef48489496e8e8a33b6a9329eda" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 46211 } } peers { permanent_uuid: "e8a36744066c43debf42fc9fa6d47efd" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38117 } }
I20260809 02:06:00.386929 19687 heartbeater.cc:503] Master 127.1.192.254:42427 was elected leader, sending a full tablet report...
I20260809 02:06:00.387154 19700 ts_tablet_manager.cc:1428] T da13c8d014e1480b93127bf1a2f3924f P f4d44812f3f84c4db2c91ba76af08a89: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
W20260809 02:06:00.559082 19688 tablet.cc:2369] T da13c8d014e1480b93127bf1a2f3924f P f4d44812f3f84c4db2c91ba76af08a89: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 02:06:00.583165 19540 tablet.cc:2369] T da13c8d014e1480b93127bf1a2f3924f P e8a36744066c43debf42fc9fa6d47efd: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 02:06:00.609663 19394 tablet.cc:2369] T da13c8d014e1480b93127bf1a2f3924f P 1a567ef48489496e8e8a33b6a9329eda: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 02:06:00.753125 19544 raft_consensus.cc:482] T da13c8d014e1480b93127bf1a2f3924f P e8a36744066c43debf42fc9fa6d47efd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:06:00.753423 19544 raft_consensus.cc:504] T da13c8d014e1480b93127bf1a2f3924f P e8a36744066c43debf42fc9fa6d47efd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4d44812f3f84c4db2c91ba76af08a89" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 40857 } } peers { permanent_uuid: "1a567ef48489496e8e8a33b6a9329eda" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 46211 } } peers { permanent_uuid: "e8a36744066c43debf42fc9fa6d47efd" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38117 } }
I20260809 02:06:00.754726 19544 leader_election.cc:288] T da13c8d014e1480b93127bf1a2f3924f P e8a36744066c43debf42fc9fa6d47efd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f4d44812f3f84c4db2c91ba76af08a89 (127.1.192.195:40857), 1a567ef48489496e8e8a33b6a9329eda (127.1.192.193:46211)
I20260809 02:06:00.760669 19705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:00.755395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.194:47781 (local address 127.1.192.193:46211)
0809 02:06:00.755792 (+ 397us) server_negotiation.cc:206] Beginning negotiation
0809 02:06:00.755795 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:06:00.755813 (+ 18us) server_negotiation.cc:407] Connection header received
0809 02:06:00.755871 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:06:00.755873 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:06:00.755955 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:06:00.756069 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:06:00.756366 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:00.757235 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:00.759104 (+ 1869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:00.759853 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:00.759995 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:00.760176 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0809 02:06:00.760439 (+ 263us) server_negotiation.cc:299] Negotiation successful
0809 02:06:00.760534 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":124,"threads_started":1}
I20260809 02:06:00.760668 19697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:00.755163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.194:51523 (local address 127.1.192.195:40857)
0809 02:06:00.755352 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0809 02:06:00.755356 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:06:00.755457 (+ 101us) server_negotiation.cc:407] Connection header received
0809 02:06:00.755737 (+ 280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:06:00.755746 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:06:00.755788 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:06:00.755826 (+ 38us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:06:00.756272 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:00.757214 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:00.759091 (+ 1877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:00.759820 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:00.759986 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:00.760046 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 02:06:00.760439 (+ 393us) server_negotiation.cc:299] Negotiation successful
0809 02:06:00.760535 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260809 02:06:00.760686 19703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:00.754951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.195:40857 (local address 127.1.192.194:51523)
0809 02:06:00.755354 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0809 02:06:00.755372 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 02:06:00.755563 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:06:00.755959 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:06:00.755961 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:06:00.755976 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:06:00.756129 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:06:00.756136 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:06:00.757391 (+ 1255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:06:00.757395 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:06:00.758895 (+ 1500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:06:00.758906 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:06:00.760014 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:06:00.760019 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:06:00.760204 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:00.760276 (+ 72us) client_negotiation.cc:769] Sending connection context
0809 02:06:00.760367 (+ 91us) client_negotiation.cc:240] Negotiation successful
0809 02:06:00.760480 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":325,"mutex_wait_us":123,"thread_start_us":126,"threads_started":1}
I20260809 02:06:00.760692 19704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:00.755168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.193:46211 (local address 127.1.192.194:47781)
0809 02:06:00.755539 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0809 02:06:00.755552 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:06:00.755625 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:06:00.756110 (+ 485us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:06:00.756112 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:06:00.756127 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:06:00.756226 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:06:00.756234 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:06:00.757391 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:06:00.757395 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:06:00.758893 (+ 1498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:06:00.758906 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:06:00.760014 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:06:00.760019 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:06:00.760193 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:00.760282 (+ 89us) client_negotiation.cc:769] Sending connection context
0809 02:06:00.760367 (+ 85us) client_negotiation.cc:240] Negotiation successful
0809 02:06:00.760480 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":314,"mutex_wait_us":180,"thread_start_us":119,"threads_started":1}
I20260809 02:06:00.761508 19343 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da13c8d014e1480b93127bf1a2f3924f" candidate_uuid: "e8a36744066c43debf42fc9fa6d47efd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a567ef48489496e8e8a33b6a9329eda" is_pre_election: true
I20260809 02:06:00.761600 19637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da13c8d014e1480b93127bf1a2f3924f" candidate_uuid: "e8a36744066c43debf42fc9fa6d47efd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4d44812f3f84c4db2c91ba76af08a89" is_pre_election: true
I20260809 02:06:00.761891 19343 raft_consensus.cc:2454] T da13c8d014e1480b93127bf1a2f3924f P 1a567ef48489496e8e8a33b6a9329eda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8a36744066c43debf42fc9fa6d47efd in term 0.
I20260809 02:06:00.761909 19637 raft_consensus.cc:2454] T da13c8d014e1480b93127bf1a2f3924f P f4d44812f3f84c4db2c91ba76af08a89 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8a36744066c43debf42fc9fa6d47efd in term 0.
I20260809 02:06:00.762452 19420 leader_election.cc:302] T da13c8d014e1480b93127bf1a2f3924f P e8a36744066c43debf42fc9fa6d47efd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 1a567ef48489496e8e8a33b6a9329eda, e8a36744066c43debf42fc9fa6d47efd, f4d44812f3f84c4db2c91ba76af08a89; no voters:
I20260809 02:06:00.762779 19544 raft_consensus.cc:2781] T da13c8d014e1480b93127bf1a2f3924f P e8a36744066c43debf42fc9fa6d47efd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:06:00.762882 19544 raft_consensus.cc:482] T da13c8d014e1480b93127bf1a2f3924f P e8a36744066c43debf42fc9fa6d47efd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:06:00.762928 19544 raft_consensus.cc:3037] T da13c8d014e1480b93127bf1a2f3924f P e8a36744066c43debf42fc9fa6d47efd [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:06:00.764114 19544 raft_consensus.cc:504] T da13c8d014e1480b93127bf1a2f3924f P e8a36744066c43debf42fc9fa6d47efd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4d44812f3f84c4db2c91ba76af08a89" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 40857 } } peers { permanent_uuid: "1a567ef48489496e8e8a33b6a9329eda" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 46211 } } peers { permanent_uuid: "e8a36744066c43debf42fc9fa6d47efd" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38117 } }
I20260809 02:06:00.764541 19544 leader_election.cc:288] T da13c8d014e1480b93127bf1a2f3924f P e8a36744066c43debf42fc9fa6d47efd [CANDIDATE]: Term 1 election: Requested vote from peers f4d44812f3f84c4db2c91ba76af08a89 (127.1.192.195:40857), 1a567ef48489496e8e8a33b6a9329eda (127.1.192.193:46211)
I20260809 02:06:00.764962 19637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da13c8d014e1480b93127bf1a2f3924f" candidate_uuid: "e8a36744066c43debf42fc9fa6d47efd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4d44812f3f84c4db2c91ba76af08a89"
I20260809 02:06:00.764943 19343 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da13c8d014e1480b93127bf1a2f3924f" candidate_uuid: "e8a36744066c43debf42fc9fa6d47efd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a567ef48489496e8e8a33b6a9329eda"
I20260809 02:06:00.765106 19637 raft_consensus.cc:3037] T da13c8d014e1480b93127bf1a2f3924f P f4d44812f3f84c4db2c91ba76af08a89 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:06:00.765106 19343 raft_consensus.cc:3037] T da13c8d014e1480b93127bf1a2f3924f P 1a567ef48489496e8e8a33b6a9329eda [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:06:00.766472 19343 raft_consensus.cc:2454] T da13c8d014e1480b93127bf1a2f3924f P 1a567ef48489496e8e8a33b6a9329eda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8a36744066c43debf42fc9fa6d47efd in term 1.
I20260809 02:06:00.766513 19637 raft_consensus.cc:2454] T da13c8d014e1480b93127bf1a2f3924f P f4d44812f3f84c4db2c91ba76af08a89 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8a36744066c43debf42fc9fa6d47efd in term 1.
I20260809 02:06:00.766923 19419 leader_election.cc:302] T da13c8d014e1480b93127bf1a2f3924f P e8a36744066c43debf42fc9fa6d47efd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e8a36744066c43debf42fc9fa6d47efd, f4d44812f3f84c4db2c91ba76af08a89; no voters:
I20260809 02:06:00.767127 19544 raft_consensus.cc:2781] T da13c8d014e1480b93127bf1a2f3924f P e8a36744066c43debf42fc9fa6d47efd [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:06:00.767311 19544 raft_consensus.cc:686] T da13c8d014e1480b93127bf1a2f3924f P e8a36744066c43debf42fc9fa6d47efd [term 1 LEADER]: Becoming Leader. State: Replica: e8a36744066c43debf42fc9fa6d47efd, State: Running, Role: LEADER
I20260809 02:06:00.767608 19544 consensus_queue.cc:237] T da13c8d014e1480b93127bf1a2f3924f P e8a36744066c43debf42fc9fa6d47efd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4d44812f3f84c4db2c91ba76af08a89" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 40857 } } peers { permanent_uuid: "1a567ef48489496e8e8a33b6a9329eda" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 46211 } } peers { permanent_uuid: "e8a36744066c43debf42fc9fa6d47efd" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38117 } }
I20260809 02:06:00.770627 19191 catalog_manager.cc:5495] T da13c8d014e1480b93127bf1a2f3924f P e8a36744066c43debf42fc9fa6d47efd reported cstate change: term changed from 0 to 1, leader changed from <none> to e8a36744066c43debf42fc9fa6d47efd (127.1.192.194). New cstate: current_term: 1 leader_uuid: "e8a36744066c43debf42fc9fa6d47efd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4d44812f3f84c4db2c91ba76af08a89" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 40857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1a567ef48489496e8e8a33b6a9329eda" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 46211 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8a36744066c43debf42fc9fa6d47efd" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38117 } health_report { overall_health: HEALTHY } } }
I20260809 02:06:00.823005 19709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:00.818160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57150 (local address 127.1.192.194:38117)
0809 02:06:00.818553 (+ 393us) server_negotiation.cc:206] Beginning negotiation
0809 02:06:00.818561 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 02:06:00.818574 (+ 13us) server_negotiation.cc:407] Connection header received
0809 02:06:00.818683 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:06:00.818688 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:06:00.818757 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:06:00.818864 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:06:00.819230 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:00.819913 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:00.820992 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:00.821248 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:00.821341 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:00.821809 (+ 468us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:06:00.822450 (+ 641us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:06:00.822578 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0809 02:06:00.822737 (+ 159us) server_negotiation.cc:299] Negotiation successful
0809 02:06:00.822800 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":266,"thread_start_us":158,"threads_started":1}
I20260809 02:06:00.827533 19343 raft_consensus.cc:1261] T da13c8d014e1480b93127bf1a2f3924f P 1a567ef48489496e8e8a33b6a9329eda [term 1 FOLLOWER]: Refusing update from remote peer e8a36744066c43debf42fc9fa6d47efd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:06:00.827538 19637 raft_consensus.cc:1261] T da13c8d014e1480b93127bf1a2f3924f P f4d44812f3f84c4db2c91ba76af08a89 [term 1 FOLLOWER]: Refusing update from remote peer e8a36744066c43debf42fc9fa6d47efd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:06:00.828233 19544 consensus_queue.cc:1035] T da13c8d014e1480b93127bf1a2f3924f P e8a36744066c43debf42fc9fa6d47efd [LEADER]: Connected to new peer: Peer: permanent_uuid: "f4d44812f3f84c4db2c91ba76af08a89" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 40857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:06:00.828636 19706 consensus_queue.cc:1035] T da13c8d014e1480b93127bf1a2f3924f P e8a36744066c43debf42fc9fa6d47efd [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a567ef48489496e8e8a33b6a9329eda" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 46211 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:06:00.835007 19710 mvcc.cc:205] Tried to move back new op lower bound from 7316443794742005760 to 7316443794506813440. Current Snapshot: MvccSnapshot[applied={T|T < 7316443794742005760}]
I20260809 02:06:00.843989 19188 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52156:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:06:00.844255 19188 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52156:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:06:00.846858 19188 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9d388ae00b2c47b78dd7ca03332c3071: Sending DeleteTablet for 3 replicas of tablet da13c8d014e1480b93127bf1a2f3924f
I20260809 02:06:00.847646 19468 tablet_service.cc:1514] Processing DeleteTablet for tablet da13c8d014e1480b93127bf1a2f3924f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:06:00 UTC) from {username='kudu', principal='kudu/127.1.192.254@KRBTEST.COM'} at 127.0.0.1:57140
I20260809 02:06:00.847723 19322 tablet_service.cc:1514] Processing DeleteTablet for tablet da13c8d014e1480b93127bf1a2f3924f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:06:00 UTC) from {username='kudu', principal='kudu/127.1.192.254@KRBTEST.COM'} at 127.0.0.1:53506
I20260809 02:06:00.847826 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 19235
I20260809 02:06:00.848529 19614 tablet_service.cc:1514] Processing DeleteTablet for tablet da13c8d014e1480b93127bf1a2f3924f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:06:00 UTC) from {username='kudu', principal='kudu/127.1.192.254@KRBTEST.COM'} at 127.0.0.1:57836
I20260809 02:06:00.849663 19721 tablet_replica.cc:330] T da13c8d014e1480b93127bf1a2f3924f P f4d44812f3f84c4db2c91ba76af08a89: stopping tablet replica
I20260809 02:06:00.850075 19721 raft_consensus.cc:2229] T da13c8d014e1480b93127bf1a2f3924f P f4d44812f3f84c4db2c91ba76af08a89 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:06:00.850456 19721 raft_consensus.cc:2258] T da13c8d014e1480b93127bf1a2f3924f P f4d44812f3f84c4db2c91ba76af08a89 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:06:00.851089 19720 tablet_replica.cc:330] T da13c8d014e1480b93127bf1a2f3924f P e8a36744066c43debf42fc9fa6d47efd: stopping tablet replica
I20260809 02:06:00.851231 19720 raft_consensus.cc:2229] T da13c8d014e1480b93127bf1a2f3924f P e8a36744066c43debf42fc9fa6d47efd [term 1 LEADER]: Raft consensus shutting down.
I20260809 02:06:00.851564 19720 raft_consensus.cc:2258] T da13c8d014e1480b93127bf1a2f3924f P e8a36744066c43debf42fc9fa6d47efd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:06:00.852519 19720 ts_tablet_manager.cc:1905] T da13c8d014e1480b93127bf1a2f3924f P e8a36744066c43debf42fc9fa6d47efd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:06:00.852962 19721 ts_tablet_manager.cc:1905] T da13c8d014e1480b93127bf1a2f3924f P f4d44812f3f84c4db2c91ba76af08a89: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:06:00.855307 19720 ts_tablet_manager.cc:1918] T da13c8d014e1480b93127bf1a2f3924f P e8a36744066c43debf42fc9fa6d47efd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:06:00.855394 19720 log.cc:1196] T da13c8d014e1480b93127bf1a2f3924f P e8a36744066c43debf42fc9fa6d47efd: Deleting WAL directory at /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-1/wal/wals/da13c8d014e1480b93127bf1a2f3924f
I20260809 02:06:00.855686 19720 ts_tablet_manager.cc:1939] T da13c8d014e1480b93127bf1a2f3924f P e8a36744066c43debf42fc9fa6d47efd: Deleting consensus metadata
I20260809 02:06:00.856316 19721 ts_tablet_manager.cc:1918] T da13c8d014e1480b93127bf1a2f3924f P f4d44812f3f84c4db2c91ba76af08a89: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:06:00.856634 19721 log.cc:1196] T da13c8d014e1480b93127bf1a2f3924f P f4d44812f3f84c4db2c91ba76af08a89: Deleting WAL directory at /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-2/wal/wals/da13c8d014e1480b93127bf1a2f3924f
I20260809 02:06:00.856626 19177 catalog_manager.cc:4841] TS e8a36744066c43debf42fc9fa6d47efd (127.1.192.194:38117): tablet da13c8d014e1480b93127bf1a2f3924f (table test-table [id=2a1788194d94427ea50afccde17fcf59]) successfully deleted
I20260809 02:06:00.857028 19721 ts_tablet_manager.cc:1939] T da13c8d014e1480b93127bf1a2f3924f P f4d44812f3f84c4db2c91ba76af08a89: Deleting consensus metadata
I20260809 02:06:00.858450 19177 catalog_manager.cc:4841] TS f4d44812f3f84c4db2c91ba76af08a89 (127.1.192.195:40857): tablet da13c8d014e1480b93127bf1a2f3924f (table test-table [id=2a1788194d94427ea50afccde17fcf59]) successfully deleted
W20260809 02:06:00.862032 19178 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.192.193:46211 (error 108)
I20260809 02:06:00.862699 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 19398
I20260809 02:06:00.863749 19723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:00.862964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.193:46211 (local address 127.0.0.1:53518)
0809 02:06:00.863348 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0809 02:06:00.863607 (+ 259us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.193:46211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":56,"threads_started":1}
W20260809 02:06:00.863894 19178 catalog_manager.cc:4584] TS 1a567ef48489496e8e8a33b6a9329eda (127.1.192.193:46211): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet da13c8d014e1480b93127bf1a2f3924f: Network error: Client connection negotiation failed: client connection to 127.1.192.193:46211: connect: Connection refused (error 111)
I20260809 02:06:00.870110 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 19543
I20260809 02:06:00.878022 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 19158
I20260809 02:06:20.886838 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.192.254:42427
--webserver_interface=127.1.192.254
--webserver_port=40169
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab
--principal=kudu/127.1.192.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:33273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.192.254:42427
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:06:21.001716 19724 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:06:21.002002 19724 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:06:21.002043 19724 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:06:21.006186 19724 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 02:06:21.006253 19724 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 02:06:21.006271 19724 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 02:06:21.006286 19724 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:06:21.006304 19724 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:06:21.006323 19724 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:06:21.006340 19724 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:06:21.006354 19724 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:06:21.011796 19724 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:33273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.1.192.254:42427
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab
--principal=kudu/127.1.192.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.192.254:42427
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.192.254
--webserver_port=40169
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19724
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:06:21.012881 19724 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:06:21.014000 19724 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 02:06:21.023248 19724 server_base.cc:1034] running on GCE node
W20260809 02:06:21.023206 19729 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:06:21.023206 19732 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:06:21.023244 19730 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:06:21.024240 19724 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:06:21.025429 19724 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:06:21.026661 19724 hybrid_clock.cc:648] HybridClock initialized: now 1786241181026620 us; error 52 us; skew 500 ppm
Aug 09 02:06:21 dist-test-slave-ss4n krb5kdc[18571](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241181, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:06:21.029747 19724 init.cc:377] Logged in from keytab as kudu/127.1.192.254@KRBTEST.COM (short username kudu)
I20260809 02:06:21.030774 19724 webserver.cc:460] Webserver started at http://127.1.192.254:40169/ using document root <none> and password file <none>
I20260809 02:06:21.031374 19724 fs_manager.cc:359] Metadata directory not provided
I20260809 02:06:21.031472 19724 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:06:21.035785 19724 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260809 02:06:21.038162 19739 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:06:21.039650 19724 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 02:06:21.039844 19724 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/master-0/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/master-0/wal
uuid: "9d388ae00b2c47b78dd7ca03332c3071"
format_stamp: "Formatted at 2026-08-09 02:05:38 on dist-test-slave-ss4n"
server_key: "531e65c25971eaa04e81ad921adb7ed0"
server_key_iv: "e6e3375925835d2446c298332cab4ecd"
server_key_version: "encryptionkey@0"
I20260809 02:06:21.040501 19724 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:06:21.056252 19724 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:06:21.059844 19724 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:06:21.060096 19724 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:06:21.068650 19724 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.254:42427
I20260809 02:06:21.068943 19792 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.254:42427 every 8 connection(s)
I20260809 02:06:21.069965 19724 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
I20260809 02:06:21.074483 19793 sys_catalog.cc:263] Verifying existing consensus state
I20260809 02:06:21.074976 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 19724
I20260809 02:06:21.075460 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.192.193:46211
--local_ip_for_outbound_sockets=127.1.192.193
--tserver_master_addrs=127.1.192.254:42427
--webserver_port=37133
--webserver_interface=127.1.192.193
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.193.keytab
--principal=kudu/127.1.192.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:33273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 02:06:21.076336 19793 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d388ae00b2c47b78dd7ca03332c3071: Bootstrap starting.
I20260809 02:06:21.094625 19793 log.cc:824] T 00000000000000000000000000000000 P 9d388ae00b2c47b78dd7ca03332c3071: Log is configured to *not* fsync() on all Append() calls
I20260809 02:06:21.106374 19793 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d388ae00b2c47b78dd7ca03332c3071: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 02:06:21.106809 19793 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d388ae00b2c47b78dd7ca03332c3071: Bootstrap complete.
I20260809 02:06:21.111040 19793 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9d388ae00b2c47b78dd7ca03332c3071 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d388ae00b2c47b78dd7ca03332c3071" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 42427 } }
I20260809 02:06:21.111486 19793 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9d388ae00b2c47b78dd7ca03332c3071 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d388ae00b2c47b78dd7ca03332c3071, State: Initialized, Role: FOLLOWER
I20260809 02:06:21.111976 19793 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9d388ae00b2c47b78dd7ca03332c3071 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d388ae00b2c47b78dd7ca03332c3071" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 42427 } }
I20260809 02:06:21.112138 19793 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9d388ae00b2c47b78dd7ca03332c3071 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:06:21.112229 19793 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9d388ae00b2c47b78dd7ca03332c3071 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:06:21.112354 19793 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9d388ae00b2c47b78dd7ca03332c3071 [term 2 FOLLOWER]: Advancing to term 3
I20260809 02:06:21.114403 19793 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9d388ae00b2c47b78dd7ca03332c3071 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d388ae00b2c47b78dd7ca03332c3071" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 42427 } }
I20260809 02:06:21.114748 19793 leader_election.cc:302] T 00000000000000000000000000000000 P 9d388ae00b2c47b78dd7ca03332c3071 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9d388ae00b2c47b78dd7ca03332c3071; no voters:
I20260809 02:06:21.115108 19793 leader_election.cc:288] T 00000000000000000000000000000000 P 9d388ae00b2c47b78dd7ca03332c3071 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260809 02:06:21.115199 19797 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9d388ae00b2c47b78dd7ca03332c3071 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 02:06:21.115483 19797 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9d388ae00b2c47b78dd7ca03332c3071 [term 3 LEADER]: Becoming Leader. State: Replica: 9d388ae00b2c47b78dd7ca03332c3071, State: Running, Role: LEADER
I20260809 02:06:21.115852 19797 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9d388ae00b2c47b78dd7ca03332c3071 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d388ae00b2c47b78dd7ca03332c3071" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 42427 } }
I20260809 02:06:21.116415 19793 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9d388ae00b2c47b78dd7ca03332c3071 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:06:21.116809 19797 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d388ae00b2c47b78dd7ca03332c3071 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9d388ae00b2c47b78dd7ca03332c3071. Latest consensus state: current_term: 3 leader_uuid: "9d388ae00b2c47b78dd7ca03332c3071" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d388ae00b2c47b78dd7ca03332c3071" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 42427 } } }
I20260809 02:06:21.117452 19798 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d388ae00b2c47b78dd7ca03332c3071 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "9d388ae00b2c47b78dd7ca03332c3071" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d388ae00b2c47b78dd7ca03332c3071" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 42427 } } }
I20260809 02:06:21.117630 19797 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d388ae00b2c47b78dd7ca03332c3071 [sys.catalog]: This master's current role is: LEADER
I20260809 02:06:21.118429 19798 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d388ae00b2c47b78dd7ca03332c3071 [sys.catalog]: This master's current role is: LEADER
I20260809 02:06:21.118602 19807 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:06:21.124330 19807 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:06:21.125136 19807 catalog_manager.cc:1259] Loaded cluster ID: 7ae31931097d4e48b0910208ee0e9f08
I20260809 02:06:21.125209 19807 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:06:21.127940 19807 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:06:21.129048 19807 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 9d388ae00b2c47b78dd7ca03332c3071: Loaded TSK: 0
I20260809 02:06:21.129166 19807 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 9d388ae00b2c47b78dd7ca03332c3071: Loaded TSK: 1
F20260809 02:06:21.130128 19807 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f307d4179ad google::LogMessage::Fail() at ??:0
@ 0x7f307d41a88a google::LogMessage::SendToLog() at ??:0
@ 0x7f307d4174de google::LogMessage::Flush() at ??:0
@ 0x7f307d418a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f30800db18d kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f30800d3f0d kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f3087d7c7bb kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f3087d5699c kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f3087d57178 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlvE3_clEv at ??:0
@ 0x7f3087d98df8 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f3087db782b std::function<>::operator()() at ??:0
@ 0x7f3087d56b4f _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f3087d57f3e kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f3087d54579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f3087d98640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f307f6693d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f307f669c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f307f66b3f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f307f65a512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f3080c9e6db start_thread at ??:0
@ 0x7f307bf1561f clone at ??:0
W20260809 02:06:21.247130 19795 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:06:21.247458 19795 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:06:21.247514 19795 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:06:21.252090 19795 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:06:21.252168 19795 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:06:21.252233 19795 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.192.193
I20260809 02:06:21.257934 19795 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:33273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.193.keytab
--principal=kudu/127.1.192.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.192.193:46211
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.192.193
--webserver_port=37133
--tserver_master_addrs=127.1.192.254:42427
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19795
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.192.193
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:06:21.259076 19795 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:06:21.260160 19795 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:06:21.271085 19843 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:06:21.271165 19846 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:06:21.271085 19844 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:06:21.271471 19795 server_base.cc:1034] running on GCE node
I20260809 02:06:21.271930 19795 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:06:21.272583 19795 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:06:21.273837 19795 hybrid_clock.cc:648] HybridClock initialized: now 1786241181273807 us; error 50 us; skew 500 ppm
Aug 09 02:06:21 dist-test-slave-ss4n krb5kdc[18571](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241181, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:06:21.276932 19795 init.cc:377] Logged in from keytab as kudu/127.1.192.193@KRBTEST.COM (short username kudu)
I20260809 02:06:21.277972 19795 webserver.cc:460] Webserver started at http://127.1.192.193:37133/ using document root <none> and password file <none>
I20260809 02:06:21.278563 19795 fs_manager.cc:359] Metadata directory not provided
I20260809 02:06:21.278661 19795 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:06:21.282836 19795 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 02:06:21.285290 19853 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:06:21.286710 19795 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 02:06:21.286854 19795 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-0/wal
uuid: "1a567ef48489496e8e8a33b6a9329eda"
format_stamp: "Formatted at 2026-08-09 02:05:38 on dist-test-slave-ss4n"
server_key: "194b68000e2f13498410a1257ea231ce"
server_key_iv: "5f1024baeffe9f33765e4e4ad219ca64"
server_key_version: "encryptionkey@0"
I20260809 02:06:21.287454 19795 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:06:21.301110 19795 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:06:21.304697 19795 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:06:21.304908 19795 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:06:21.305579 19795 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:06:21.306723 19860 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 02:06:21.312388 19795 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 02:06:21.312511 19795 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20260809 02:06:21.312598 19795 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 02:06:21.314678 19795 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 02:06:21.314759 19795 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20260809 02:06:21.315121 19860 tablet_bootstrap.cc:492] T da13c8d014e1480b93127bf1a2f3924f P 1a567ef48489496e8e8a33b6a9329eda: Bootstrap starting.
I20260809 02:06:21.326287 19795 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.193:46211
I20260809 02:06:21.326562 19972 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.193:46211 every 8 connection(s)
I20260809 02:06:21.327454 19795 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-0/data/info.pb
I20260809 02:06:21.334632 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 19795
I20260809 02:06:21.335139 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.192.194:38117
--local_ip_for_outbound_sockets=127.1.192.194
--tserver_master_addrs=127.1.192.254:42427
--webserver_port=36429
--webserver_interface=127.1.192.194
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.194.keytab
--principal=kudu/127.1.192.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:33273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 02:06:21.338150 19860 log.cc:824] T da13c8d014e1480b93127bf1a2f3924f P 1a567ef48489496e8e8a33b6a9329eda: Log is configured to *not* fsync() on all Append() calls
I20260809 02:06:21.345691 19860 tablet_bootstrap.cc:492] T da13c8d014e1480b93127bf1a2f3924f P 1a567ef48489496e8e8a33b6a9329eda: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 02:06:21.346122 19860 tablet_bootstrap.cc:492] T da13c8d014e1480b93127bf1a2f3924f P 1a567ef48489496e8e8a33b6a9329eda: Bootstrap complete.
I20260809 02:06:21.347337 19860 ts_tablet_manager.cc:1397] T da13c8d014e1480b93127bf1a2f3924f P 1a567ef48489496e8e8a33b6a9329eda: Time spent bootstrapping tablet: real 0.032s user 0.018s sys 0.013s
I20260809 02:06:21.349954 19860 raft_consensus.cc:348] T da13c8d014e1480b93127bf1a2f3924f P 1a567ef48489496e8e8a33b6a9329eda [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4d44812f3f84c4db2c91ba76af08a89" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 40857 } } peers { permanent_uuid: "1a567ef48489496e8e8a33b6a9329eda" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 46211 } } peers { permanent_uuid: "e8a36744066c43debf42fc9fa6d47efd" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38117 } }
I20260809 02:06:21.350462 19860 raft_consensus.cc:729] T da13c8d014e1480b93127bf1a2f3924f P 1a567ef48489496e8e8a33b6a9329eda [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a567ef48489496e8e8a33b6a9329eda, State: Initialized, Role: FOLLOWER
I20260809 02:06:21.351025 19860 consensus_queue.cc:260] T da13c8d014e1480b93127bf1a2f3924f P 1a567ef48489496e8e8a33b6a9329eda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4d44812f3f84c4db2c91ba76af08a89" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 40857 } } peers { permanent_uuid: "1a567ef48489496e8e8a33b6a9329eda" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 46211 } } peers { permanent_uuid: "e8a36744066c43debf42fc9fa6d47efd" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38117 } }
I20260809 02:06:21.352142 19860 ts_tablet_manager.cc:1428] T da13c8d014e1480b93127bf1a2f3924f P 1a567ef48489496e8e8a33b6a9329eda: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
W20260809 02:06:21.387161 19975 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:06:21.329661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:42427 (local address 127.1.192.193:43247)
0809 02:06:21.330173 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0809 02:06:21.330232 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0809 02:06:21.331094 (+ 862us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:06:21.386349 (+ 55255us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:42427: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":151,"threads_started":1}
W20260809 02:06:21.387822 19973 heartbeater.cc:644] Failed to heartbeat to 127.1.192.254:42427 (0 consecutive failures): Network error: Failed to ping master at 127.1.192.254:42427: Client connection negotiation failed: client connection to 127.1.192.254:42427: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 02:06:21.388695 19975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:21.388378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:42427 (local address 127.1.192.193:47669)
0809 02:06:21.388529 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0809 02:06:21.388624 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:42427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260809 02:06:21.389657 19975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:21.389284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:42427 (local address 127.1.192.193:53301)
0809 02:06:21.389414 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0809 02:06:21.389474 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:42427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260809 02:06:21.389922 19973 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 02:06:21.467316 19976 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:06:21.467592 19976 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:06:21.467639 19976 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:06:21.471757 19976 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:06:21.471829 19976 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:06:21.471891 19976 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.192.194
I20260809 02:06:21.477373 19976 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:33273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.194.keytab
--principal=kudu/127.1.192.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.192.194:38117
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.192.194
--webserver_port=36429
--tserver_master_addrs=127.1.192.254:42427
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19976
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.192.194
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:06:21.478574 19976 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:06:21.479606 19976 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:06:21.489837 19985 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:06:21.489998 19976 server_base.cc:1034] running on GCE node
W20260809 02:06:21.489837 19987 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:06:21.489842 19983 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:06:21.490679 19976 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:06:21.491369 19976 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:06:21.492555 19976 hybrid_clock.cc:648] HybridClock initialized: now 1786241181492517 us; error 77 us; skew 500 ppm
Aug 09 02:06:21 dist-test-slave-ss4n krb5kdc[18571](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241181, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:06:21.495568 19976 init.cc:377] Logged in from keytab as kudu/127.1.192.194@KRBTEST.COM (short username kudu)
I20260809 02:06:21.496488 19976 webserver.cc:460] Webserver started at http://127.1.192.194:36429/ using document root <none> and password file <none>
I20260809 02:06:21.497021 19976 fs_manager.cc:359] Metadata directory not provided
I20260809 02:06:21.497092 19976 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:06:21.500973 19976 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260809 02:06:21.503298 19994 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:06:21.504813 19976 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 02:06:21.504930 19976 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-1/wal
uuid: "e8a36744066c43debf42fc9fa6d47efd"
format_stamp: "Formatted at 2026-08-09 02:05:38 on dist-test-slave-ss4n"
server_key: "8214780cf82e86dfdcc54772a3e6d20a"
server_key_iv: "c4fe70c99d60c3a0f7122d7260b3a121"
server_key_version: "encryptionkey@0"
I20260809 02:06:21.505843 19976 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:06:21.537973 19976 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:06:21.541903 19976 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:06:21.542099 19976 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:06:21.542727 19976 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:06:21.543613 19976 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:06:21.543666 19976 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:06:21.543710 19976 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:06:21.543728 19976 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:06:21.554512 19976 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.194:38117
I20260809 02:06:21.554541 20116 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.194:38117 every 8 connection(s)
I20260809 02:06:21.555708 19976 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-1/data/info.pb
I20260809 02:06:21.559612 20119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:21.557982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:42427 (local address 127.1.192.194:53397)
0809 02:06:21.558746 (+ 764us) negotiation.cc:107] Waiting for socket to connect
0809 02:06:21.558863 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:42427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":559,"thread_start_us":116,"threads_started":1}
W20260809 02:06:21.560091 20117 heartbeater.cc:644] Failed to heartbeat to 127.1.192.254:42427 (0 consecutive failures): Network error: Failed to ping master at 127.1.192.254:42427: Client connection negotiation failed: client connection to 127.1.192.254:42427: connect: Connection refused (error 111)
I20260809 02:06:21.560781 20119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:21.560545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:42427 (local address 127.1.192.194:46699)
0809 02:06:21.560669 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0809 02:06:21.560721 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:42427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260809 02:06:21.561625 20119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:21.561284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:42427 (local address 127.1.192.194:46153)
0809 02:06:21.561416 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0809 02:06:21.561467 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:42427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260809 02:06:21.561825 20117 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 02:06:21.565299 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 19976
I20260809 02:06:21.565851 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.192.195:40857
--local_ip_for_outbound_sockets=127.1.192.195
--tserver_master_addrs=127.1.192.254:42427
--webserver_port=41081
--webserver_interface=127.1.192.195
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.195.keytab
--principal=kudu/127.1.192.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:33273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:06:21.683383 20120 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:06:21.683735 20120 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:06:21.683826 20120 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:06:21.688467 20120 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:06:21.688637 20120 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:06:21.688737 20120 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.192.195
I20260809 02:06:21.695179 20120 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:33273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.195.keytab
--principal=kudu/127.1.192.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.192.195:40857
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.192.195
--webserver_port=41081
--tserver_master_addrs=127.1.192.254:42427
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20120
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.192.195
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:06:21.696455 20120 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:06:21.697721 20120 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:06:21.708379 20125 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:06:21.708666 20120 server_base.cc:1034] running on GCE node
W20260809 02:06:21.708384 20126 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:06:21.708421 20128 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:06:21.709554 20120 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:06:21.710397 20120 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:06:21.711640 20120 hybrid_clock.cc:648] HybridClock initialized: now 1786241181711593 us; error 51 us; skew 500 ppm
Aug 09 02:06:21 dist-test-slave-ss4n krb5kdc[18571](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241181, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:06:21.715152 20120 init.cc:377] Logged in from keytab as kudu/127.1.192.195@KRBTEST.COM (short username kudu)
I20260809 02:06:21.716188 20120 webserver.cc:460] Webserver started at http://127.1.192.195:41081/ using document root <none> and password file <none>
I20260809 02:06:21.716812 20120 fs_manager.cc:359] Metadata directory not provided
I20260809 02:06:21.716881 20120 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:06:21.721096 20120 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260809 02:06:21.723578 20136 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:06:21.725010 20120 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 02:06:21.725167 20120 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-2/wal
uuid: "f4d44812f3f84c4db2c91ba76af08a89"
format_stamp: "Formatted at 2026-08-09 02:05:38 on dist-test-slave-ss4n"
server_key: "9bdb0b28e7ab8d3776b4a2c64ecaba1d"
server_key_iv: "6f139f9db5bf58e0ea8fd14d29fb5ccd"
server_key_version: "encryptionkey@0"
I20260809 02:06:21.725875 20120 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:06:21.741961 20120 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:06:21.745720 20120 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:06:21.745975 20120 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:06:21.746665 20120 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:06:21.747618 20120 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:06:21.747681 20120 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:06:21.747728 20120 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:06:21.747793 20120 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:06:21.758684 20120 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.195:40857
I20260809 02:06:21.758706 20257 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.195:40857 every 8 connection(s)
I20260809 02:06:21.759750 20120 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786240941968412-1795-0/minicluster-data/ts-2/data/info.pb
I20260809 02:06:21.763813 20261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:21.761964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:42427 (local address 127.1.192.195:58491)
0809 02:06:21.762786 (+ 822us) negotiation.cc:107] Waiting for socket to connect
0809 02:06:21.762908 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:42427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":610,"thread_start_us":123,"threads_started":1}
I20260809 02:06:21.764173 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 20120
W20260809 02:06:21.764374 20258 heartbeater.cc:644] Failed to heartbeat to 127.1.192.254:42427 (0 consecutive failures): Network error: Failed to ping master at 127.1.192.254:42427: Client connection negotiation failed: client connection to 127.1.192.254:42427: connect: Connection refused (error 111)
I20260809 02:06:21.765101 20261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:21.764851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:42427 (local address 127.1.192.195:49617)
0809 02:06:21.764971 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0809 02:06:21.765027 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:42427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260809 02:06:21.765969 20261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:21.765721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:42427 (local address 127.1.192.195:52517)
0809 02:06:21.765851 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0809 02:06:21.765903 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:42427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260809 02:06:21.766213 20258 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 02:06:22.391106 20263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:22.390599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:42427 (local address 127.1.192.193:55849)
0809 02:06:22.390913 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0809 02:06:22.390975 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:42427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":78,"threads_started":1}
I20260809 02:06:22.563122 20264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:22.562589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:42427 (local address 127.1.192.194:59039)
0809 02:06:22.562923 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0809 02:06:22.562998 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:42427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":88,"threads_started":1}
I20260809 02:06:22.767392 20266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:22.766928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:42427 (local address 127.1.192.195:42713)
0809 02:06:22.767212 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0809 02:06:22.767277 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:42427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":96,"threads_started":1}
I20260809 02:06:22.922856 20267 raft_consensus.cc:482] T da13c8d014e1480b93127bf1a2f3924f P 1a567ef48489496e8e8a33b6a9329eda [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:06:22.923064 20267 raft_consensus.cc:504] T da13c8d014e1480b93127bf1a2f3924f P 1a567ef48489496e8e8a33b6a9329eda [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4d44812f3f84c4db2c91ba76af08a89" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 40857 } } peers { permanent_uuid: "1a567ef48489496e8e8a33b6a9329eda" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 46211 } } peers { permanent_uuid: "e8a36744066c43debf42fc9fa6d47efd" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38117 } }
I20260809 02:06:22.924248 20267 leader_election.cc:288] T da13c8d014e1480b93127bf1a2f3924f P 1a567ef48489496e8e8a33b6a9329eda [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f4d44812f3f84c4db2c91ba76af08a89 (127.1.192.195:40857), e8a36744066c43debf42fc9fa6d47efd (127.1.192.194:38117)
Aug 09 02:06:22 dist-test-slave-ss4n krb5kdc[18571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241181, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.193@KRBTEST.COM for kudu/127.1.192.194@KRBTEST.COM
Aug 09 02:06:22 dist-test-slave-ss4n krb5kdc[18571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241181, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.193@KRBTEST.COM for kudu/127.1.192.195@KRBTEST.COM
I20260809 02:06:22.939931 20269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:22.924740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.193:56231 (local address 127.1.192.195:40857)
0809 02:06:22.925406 (+ 666us) server_negotiation.cc:206] Beginning negotiation
0809 02:06:22.925438 (+ 32us) server_negotiation.cc:399] Waiting for connection header
0809 02:06:22.925522 (+ 84us) server_negotiation.cc:407] Connection header received
0809 02:06:22.926567 (+ 1045us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:06:22.926575 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:06:22.927072 (+ 497us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:06:22.927443 (+ 371us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:06:22.928567 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:22.929667 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:22.931014 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:22.931481 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:22.931663 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:22.934933 (+ 3270us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:06:22.934941 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:06:22.934957 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:06:22.934995 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:06:22.937327 (+ 2332us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:06:22.937830 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:06:22.937836 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:06:22.937841 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:06:22.937967 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:06:22.938265 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:06:22.938268 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:06:22.938270 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:06:22.938698 (+ 428us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:06:22.938843 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0809 02:06:22.939437 (+ 594us) server_negotiation.cc:299] Negotiation successful
0809 02:06:22.939708 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":522,"thread_start_us":103,"threads_started":1}
I20260809 02:06:22.939956 20271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:22.925168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.193:34649 (local address 127.1.192.194:38117)
0809 02:06:22.925656 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0809 02:06:22.925680 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0809 02:06:22.925714 (+ 34us) server_negotiation.cc:407] Connection header received
0809 02:06:22.926564 (+ 850us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:06:22.926575 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:06:22.927071 (+ 496us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:06:22.927443 (+ 372us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:06:22.928556 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:22.929816 (+ 1260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:22.930989 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:22.931520 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:22.931714 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:22.934705 (+ 2991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:06:22.934711 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:06:22.934724 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:06:22.934757 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:06:22.937031 (+ 2274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:06:22.937693 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:06:22.937708 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:06:22.937712 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:06:22.937844 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:06:22.938257 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:06:22.938262 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:06:22.938264 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:06:22.938700 (+ 436us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:06:22.938851 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0809 02:06:22.939455 (+ 604us) server_negotiation.cc:299] Negotiation successful
0809 02:06:22.939751 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":119,"threads_started":1}
I20260809 02:06:22.939942 20270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:22.924953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.194:38117 (local address 127.1.192.193:34649)
0809 02:06:22.925266 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0809 02:06:22.925280 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 02:06:22.925384 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:06:22.927972 (+ 2588us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:06:22.927975 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:06:22.928165 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:06:22.928347 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:06:22.928362 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:06:22.929996 (+ 1634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:06:22.929998 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:06:22.930855 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:06:22.930866 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:06:22.931662 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:06:22.931666 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:06:22.931830 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:22.932637 (+ 807us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:06:22.932651 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:06:22.934416 (+ 1765us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:06:22.937173 (+ 2757us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:06:22.937179 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:06:22.937195 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:06:22.937543 (+ 348us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:06:22.937977 (+ 434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:06:22.937980 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:06:22.937982 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:06:22.938147 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:06:22.938835 (+ 688us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:06:22.938838 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:06:22.939127 (+ 289us) client_negotiation.cc:769] Sending connection context
0809 02:06:22.939329 (+ 202us) client_negotiation.cc:240] Negotiation successful
0809 02:06:22.939725 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"mutex_wait_us":181,"spinlock_wait_cycles":13824,"thread_start_us":84,"threads_started":1}
I20260809 02:06:22.939980 20268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:22.924540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.195:40857 (local address 127.1.192.193:56231)
0809 02:06:22.924850 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0809 02:06:22.924870 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 02:06:22.924998 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:06:22.927503 (+ 2505us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:06:22.927509 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:06:22.927898 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:06:22.928347 (+ 449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:06:22.928362 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:06:22.929811 (+ 1449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:06:22.929814 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:06:22.930881 (+ 1067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:06:22.930892 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:06:22.931626 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:06:22.931629 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:06:22.931826 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:22.932559 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:06:22.932579 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:06:22.934705 (+ 2126us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:06:22.937454 (+ 2749us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:06:22.937458 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:06:22.937460 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:06:22.937702 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:06:22.938068 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:06:22.938073 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:06:22.938075 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:06:22.938162 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:06:22.938830 (+ 668us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:06:22.938837 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:06:22.939129 (+ 292us) client_negotiation.cc:769] Sending connection context
0809 02:06:22.939328 (+ 199us) client_negotiation.cc:240] Negotiation successful
0809 02:06:22.939813 (+ 485us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214,"mutex_wait_us":223,"spinlock_wait_cycles":38016,"thread_start_us":94,"threads_started":1}
W20260809 02:06:22.942708 19855 leader_election.cc:341] T da13c8d014e1480b93127bf1a2f3924f P 1a567ef48489496e8e8a33b6a9329eda [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e8a36744066c43debf42fc9fa6d47efd (127.1.192.194:38117): Not found: Tablet not found: da13c8d014e1480b93127bf1a2f3924f
W20260809 02:06:22.942957 19855 leader_election.cc:341] T da13c8d014e1480b93127bf1a2f3924f P 1a567ef48489496e8e8a33b6a9329eda [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f4d44812f3f84c4db2c91ba76af08a89 (127.1.192.195:40857): Not found: Tablet not found: da13c8d014e1480b93127bf1a2f3924f
I20260809 02:06:22.943037 19855 leader_election.cc:302] T da13c8d014e1480b93127bf1a2f3924f P 1a567ef48489496e8e8a33b6a9329eda [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1a567ef48489496e8e8a33b6a9329eda; no voters: e8a36744066c43debf42fc9fa6d47efd, f4d44812f3f84c4db2c91ba76af08a89
I20260809 02:06:22.943349 20267 raft_consensus.cc:2726] T da13c8d014e1480b93127bf1a2f3924f P 1a567ef48489496e8e8a33b6a9329eda [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 02:06:23.392428 20270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:23.392016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:42427 (local address 127.1.192.193:55299)
0809 02:06:23.392264 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0809 02:06:23.392337 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:42427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142}
I20260809 02:06:23.564663 20272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:23.564129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:42427 (local address 127.1.192.194:41851)
0809 02:06:23.564427 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0809 02:06:23.564489 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:42427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":100,"threads_started":1}
I20260809 02:06:23.768752 20273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:23.768290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:42427 (local address 127.1.192.195:43895)
0809 02:06:23.768582 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0809 02:06:23.768650 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:42427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":78,"threads_started":1}
I20260809 02:06:24.393985 20274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:24.393452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:42427 (local address 127.1.192.193:46487)
0809 02:06:24.393780 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0809 02:06:24.393851 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:42427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":88,"threads_started":1}
I20260809 02:06:24.566109 20275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:24.565643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:42427 (local address 127.1.192.194:55001)
0809 02:06:24.565936 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0809 02:06:24.566004 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:42427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":97,"threads_started":1}
I20260809 02:06:24.770412 20276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:24.769907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:42427 (local address 127.1.192.195:51257)
0809 02:06:24.770219 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0809 02:06:24.770285 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:42427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":94,"threads_started":1}
I20260809 02:06:24.852919 20277 raft_consensus.cc:482] T da13c8d014e1480b93127bf1a2f3924f P 1a567ef48489496e8e8a33b6a9329eda [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:06:24.853053 20277 raft_consensus.cc:504] T da13c8d014e1480b93127bf1a2f3924f P 1a567ef48489496e8e8a33b6a9329eda [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4d44812f3f84c4db2c91ba76af08a89" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 40857 } } peers { permanent_uuid: "1a567ef48489496e8e8a33b6a9329eda" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 46211 } } peers { permanent_uuid: "e8a36744066c43debf42fc9fa6d47efd" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38117 } }
I20260809 02:06:24.853587 20277 leader_election.cc:288] T da13c8d014e1480b93127bf1a2f3924f P 1a567ef48489496e8e8a33b6a9329eda [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f4d44812f3f84c4db2c91ba76af08a89 (127.1.192.195:40857), e8a36744066c43debf42fc9fa6d47efd (127.1.192.194:38117)
W20260809 02:06:24.854568 19855 leader_election.cc:341] T da13c8d014e1480b93127bf1a2f3924f P 1a567ef48489496e8e8a33b6a9329eda [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f4d44812f3f84c4db2c91ba76af08a89 (127.1.192.195:40857): Not found: Tablet not found: da13c8d014e1480b93127bf1a2f3924f
W20260809 02:06:24.854729 19855 leader_election.cc:341] T da13c8d014e1480b93127bf1a2f3924f P 1a567ef48489496e8e8a33b6a9329eda [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e8a36744066c43debf42fc9fa6d47efd (127.1.192.194:38117): Not found: Tablet not found: da13c8d014e1480b93127bf1a2f3924f
I20260809 02:06:24.854836 19855 leader_election.cc:302] T da13c8d014e1480b93127bf1a2f3924f P 1a567ef48489496e8e8a33b6a9329eda [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1a567ef48489496e8e8a33b6a9329eda; no voters: e8a36744066c43debf42fc9fa6d47efd, f4d44812f3f84c4db2c91ba76af08a89
I20260809 02:06:24.855070 20277 raft_consensus.cc:2726] T da13c8d014e1480b93127bf1a2f3924f P 1a567ef48489496e8e8a33b6a9329eda [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 02:06:25.395411 20278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:25.394890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:42427 (local address 127.1.192.193:56803)
0809 02:06:25.395239 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0809 02:06:25.395311 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:42427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":84,"threads_started":1}
I20260809 02:06:25.567445 20279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:25.566983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:42427 (local address 127.1.192.194:45775)
0809 02:06:25.567281 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0809 02:06:25.567343 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:42427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":79,"threads_started":1}
I20260809 02:06:25.771890 20280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:25.771371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:42427 (local address 127.1.192.195:55603)
0809 02:06:25.771725 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0809 02:06:25.771791 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:42427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":114,"threads_started":1}
I20260809 02:06:26.397029 20281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:26.396549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:42427 (local address 127.1.192.193:48859)
0809 02:06:26.396858 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0809 02:06:26.396925 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:42427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":79,"threads_started":1}
I20260809 02:06:26.568845 20282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:26.568397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:42427 (local address 127.1.192.194:47859)
0809 02:06:26.568676 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0809 02:06:26.568733 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:42427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":82,"threads_started":1}
I20260809 02:06:26.773469 20283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:26.772940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:42427 (local address 127.1.192.195:39897)
0809 02:06:26.773285 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0809 02:06:26.773357 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:42427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":86,"threads_started":1}
I20260809 02:06:27.091281 20284 raft_consensus.cc:482] T da13c8d014e1480b93127bf1a2f3924f P 1a567ef48489496e8e8a33b6a9329eda [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:06:27.091452 20284 raft_consensus.cc:504] T da13c8d014e1480b93127bf1a2f3924f P 1a567ef48489496e8e8a33b6a9329eda [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4d44812f3f84c4db2c91ba76af08a89" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 40857 } } peers { permanent_uuid: "1a567ef48489496e8e8a33b6a9329eda" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 46211 } } peers { permanent_uuid: "e8a36744066c43debf42fc9fa6d47efd" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38117 } }
I20260809 02:06:27.092032 20284 leader_election.cc:288] T da13c8d014e1480b93127bf1a2f3924f P 1a567ef48489496e8e8a33b6a9329eda [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f4d44812f3f84c4db2c91ba76af08a89 (127.1.192.195:40857), e8a36744066c43debf42fc9fa6d47efd (127.1.192.194:38117)
W20260809 02:06:27.093124 19855 leader_election.cc:341] T da13c8d014e1480b93127bf1a2f3924f P 1a567ef48489496e8e8a33b6a9329eda [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f4d44812f3f84c4db2c91ba76af08a89 (127.1.192.195:40857): Not found: Tablet not found: da13c8d014e1480b93127bf1a2f3924f
W20260809 02:06:27.093290 19855 leader_election.cc:341] T da13c8d014e1480b93127bf1a2f3924f P 1a567ef48489496e8e8a33b6a9329eda [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e8a36744066c43debf42fc9fa6d47efd (127.1.192.194:38117): Not found: Tablet not found: da13c8d014e1480b93127bf1a2f3924f
I20260809 02:06:27.093356 19855 leader_election.cc:302] T da13c8d014e1480b93127bf1a2f3924f P 1a567ef48489496e8e8a33b6a9329eda [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1a567ef48489496e8e8a33b6a9329eda; no voters: e8a36744066c43debf42fc9fa6d47efd, f4d44812f3f84c4db2c91ba76af08a89
I20260809 02:06:27.093605 20284 raft_consensus.cc:2726] T da13c8d014e1480b93127bf1a2f3924f P 1a567ef48489496e8e8a33b6a9329eda [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 02:06:27.398475 20285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:27.398032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:42427 (local address 127.1.192.193:44619)
0809 02:06:27.398312 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0809 02:06:27.398371 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:42427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":83,"threads_started":1}
I20260809 02:06:27.570251 20286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:27.569809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:42427 (local address 127.1.192.194:60633)
0809 02:06:27.570080 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0809 02:06:27.570150 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:42427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":80,"threads_started":1}
I20260809 02:06:27.775034 20287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:27.774526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:42427 (local address 127.1.192.195:57011)
0809 02:06:27.774862 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0809 02:06:27.774932 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:42427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":86,"threads_started":1}
I20260809 02:06:28.399731 20288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:28.399292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:42427 (local address 127.1.192.193:47487)
0809 02:06:28.399566 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0809 02:06:28.399630 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:42427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":81,"threads_started":1}
I20260809 02:06:28.571686 20289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:28.571107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:42427 (local address 127.1.192.194:55265)
0809 02:06:28.571499 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0809 02:06:28.571572 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:42427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":98,"threads_started":1}
I20260809 02:06:28.776607 20290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:28.776070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:42427 (local address 127.1.192.195:34015)
0809 02:06:28.776420 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0809 02:06:28.776489 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:42427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":79,"threads_started":1}
I20260809 02:06:28.915666 20291 raft_consensus.cc:482] T da13c8d014e1480b93127bf1a2f3924f P 1a567ef48489496e8e8a33b6a9329eda [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:06:28.915829 20291 raft_consensus.cc:504] T da13c8d014e1480b93127bf1a2f3924f P 1a567ef48489496e8e8a33b6a9329eda [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4d44812f3f84c4db2c91ba76af08a89" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 40857 } } peers { permanent_uuid: "1a567ef48489496e8e8a33b6a9329eda" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 46211 } } peers { permanent_uuid: "e8a36744066c43debf42fc9fa6d47efd" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38117 } }
I20260809 02:06:28.916396 20291 leader_election.cc:288] T da13c8d014e1480b93127bf1a2f3924f P 1a567ef48489496e8e8a33b6a9329eda [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f4d44812f3f84c4db2c91ba76af08a89 (127.1.192.195:40857), e8a36744066c43debf42fc9fa6d47efd (127.1.192.194:38117)
W20260809 02:06:28.917369 19855 leader_election.cc:341] T da13c8d014e1480b93127bf1a2f3924f P 1a567ef48489496e8e8a33b6a9329eda [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f4d44812f3f84c4db2c91ba76af08a89 (127.1.192.195:40857): Not found: Tablet not found: da13c8d014e1480b93127bf1a2f3924f
W20260809 02:06:28.917604 19855 leader_election.cc:341] T da13c8d014e1480b93127bf1a2f3924f P 1a567ef48489496e8e8a33b6a9329eda [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e8a36744066c43debf42fc9fa6d47efd (127.1.192.194:38117): Not found: Tablet not found: da13c8d014e1480b93127bf1a2f3924f
I20260809 02:06:28.917673 19855 leader_election.cc:302] T da13c8d014e1480b93127bf1a2f3924f P 1a567ef48489496e8e8a33b6a9329eda [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1a567ef48489496e8e8a33b6a9329eda; no voters: e8a36744066c43debf42fc9fa6d47efd, f4d44812f3f84c4db2c91ba76af08a89
I20260809 02:06:28.917815 20291 raft_consensus.cc:2726] T da13c8d014e1480b93127bf1a2f3924f P 1a567ef48489496e8e8a33b6a9329eda [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 02:06:29.401142 20293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:29.400653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:42427 (local address 127.1.192.193:38537)
0809 02:06:29.400976 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0809 02:06:29.401041 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:42427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":105,"threads_started":1}
I20260809 02:06:29.573009 20294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:29.572563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:42427 (local address 127.1.192.194:54593)
0809 02:06:29.572853 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0809 02:06:29.572910 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:42427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":100,"threads_started":1}
I20260809 02:06:29.778295 20295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:29.777784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:42427 (local address 127.1.192.195:44399)
0809 02:06:29.778112 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0809 02:06:29.778184 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:42427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":105,"threads_started":1}
I20260809 02:06:30.402631 20296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:30.402060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:42427 (local address 127.1.192.193:56035)
0809 02:06:30.402417 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0809 02:06:30.402500 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:42427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":94,"threads_started":1}
I20260809 02:06:30.574548 20297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:30.574077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:42427 (local address 127.1.192.194:39843)
0809 02:06:30.574372 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0809 02:06:30.574440 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:42427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":88,"threads_started":1}
I20260809 02:06:30.779738 20298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:30.779184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:42427 (local address 127.1.192.195:38461)
0809 02:06:30.779559 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0809 02:06:30.779631 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:42427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":85,"threads_started":1}
I20260809 02:06:30.828370 20299 raft_consensus.cc:482] T da13c8d014e1480b93127bf1a2f3924f P 1a567ef48489496e8e8a33b6a9329eda [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:06:30.828516 20299 raft_consensus.cc:504] T da13c8d014e1480b93127bf1a2f3924f P 1a567ef48489496e8e8a33b6a9329eda [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4d44812f3f84c4db2c91ba76af08a89" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 40857 } } peers { permanent_uuid: "1a567ef48489496e8e8a33b6a9329eda" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 46211 } } peers { permanent_uuid: "e8a36744066c43debf42fc9fa6d47efd" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 38117 } }
I20260809 02:06:30.828979 20299 leader_election.cc:288] T da13c8d014e1480b93127bf1a2f3924f P 1a567ef48489496e8e8a33b6a9329eda [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f4d44812f3f84c4db2c91ba76af08a89 (127.1.192.195:40857), e8a36744066c43debf42fc9fa6d47efd (127.1.192.194:38117)
W20260809 02:06:30.829993 19855 leader_election.cc:341] T da13c8d014e1480b93127bf1a2f3924f P 1a567ef48489496e8e8a33b6a9329eda [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e8a36744066c43debf42fc9fa6d47efd (127.1.192.194:38117): Not found: Tablet not found: da13c8d014e1480b93127bf1a2f3924f
W20260809 02:06:30.830192 19855 leader_election.cc:341] T da13c8d014e1480b93127bf1a2f3924f P 1a567ef48489496e8e8a33b6a9329eda [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f4d44812f3f84c4db2c91ba76af08a89 (127.1.192.195:40857): Not found: Tablet not found: da13c8d014e1480b93127bf1a2f3924f
I20260809 02:06:30.830268 19855 leader_election.cc:302] T da13c8d014e1480b93127bf1a2f3924f P 1a567ef48489496e8e8a33b6a9329eda [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1a567ef48489496e8e8a33b6a9329eda; no voters: e8a36744066c43debf42fc9fa6d47efd, f4d44812f3f84c4db2c91ba76af08a89
I20260809 02:06:30.830497 20299 raft_consensus.cc:2726] T da13c8d014e1480b93127bf1a2f3924f P 1a567ef48489496e8e8a33b6a9329eda [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 02:06:31.403993 20300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:31.403514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:42427 (local address 127.1.192.193:59481)
0809 02:06:31.403832 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0809 02:06:31.403894 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:42427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":79,"threads_started":1}
I20260809 02:06:31.575968 20301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:31.575506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:42427 (local address 127.1.192.194:40363)
0809 02:06:31.575796 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0809 02:06:31.575865 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:42427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":75,"threads_started":1}
I20260809 02:06:31.781149 20302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:31.780679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:42427 (local address 127.1.192.195:51841)
0809 02:06:31.780983 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0809 02:06:31.781045 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:42427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":90,"threads_started":1}
I20260809 02:06:32.405321 20303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:32.404882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:42427 (local address 127.1.192.193:40373)
0809 02:06:32.405163 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0809 02:06:32.405224 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:42427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":103,"threads_started":1}
I20260809 02:06:32.577379 20304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:32.576897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:42427 (local address 127.1.192.194:36039)
0809 02:06:32.577212 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0809 02:06:32.577278 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:42427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":91,"threads_started":1}
I20260809 02:06:32.782757 20305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:32.782230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:42427 (local address 127.1.192.195:39573)
0809 02:06:32.782567 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0809 02:06:32.782649 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:42427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":108,"threads_started":1}
I20260809 02:06:33.406813 20306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:33.406224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:42427 (local address 127.1.192.193:55401)
0809 02:06:33.406626 (+ 402us) negotiation.cc:107] Waiting for socket to connect
0809 02:06:33.406705 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:42427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":87,"threads_started":1}
I20260809 02:06:33.578948 20307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:33.578368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:42427 (local address 127.1.192.194:39659)
0809 02:06:33.578757 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0809 02:06:33.578840 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:42427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":97,"threads_started":1}
I20260809 02:06:33.784348 20308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:33.783855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:42427 (local address 127.1.192.195:55009)
0809 02:06:33.784182 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0809 02:06:33.784246 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:42427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":95,"threads_started":1}
I20260809 02:06:34.408313 20309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:34.407813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:42427 (local address 127.1.192.193:38767)
0809 02:06:34.408137 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0809 02:06:34.408209 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:42427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":83,"threads_started":1}
I20260809 02:06:34.580313 20310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:34.579877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:42427 (local address 127.1.192.194:41877)
0809 02:06:34.580155 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0809 02:06:34.580215 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:42427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":81,"threads_started":1}
I20260809 02:06:34.785715 20311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:34.785256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:42427 (local address 127.1.192.195:50459)
0809 02:06:34.785537 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0809 02:06:34.785613 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:42427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":81,"threads_started":1}
I20260809 02:06:35.409711 20312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:35.409236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:42427 (local address 127.1.192.193:43825)
0809 02:06:35.409552 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0809 02:06:35.409612 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:42427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":121,"threads_started":1}
I20260809 02:06:35.581995 20313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:35.581473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:42427 (local address 127.1.192.194:39661)
0809 02:06:35.581828 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0809 02:06:35.581891 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:42427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":116,"threads_started":1}
I20260809 02:06:35.787081 20314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:35.786583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:42427 (local address 127.1.192.195:52777)
0809 02:06:35.786912 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0809 02:06:35.786977 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:42427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":109,"threads_started":1}
I20260809 02:06:36.411290 20315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:36.410760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:42427 (local address 127.1.192.193:57195)
0809 02:06:36.411118 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0809 02:06:36.411183 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:42427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":157,"threads_started":1}
I20260809 02:06:36.583424 20317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:36.582978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:42427 (local address 127.1.192.194:44053)
0809 02:06:36.583261 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0809 02:06:36.583325 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.254:42427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":75,"threads_started":1}
I20260809 02:06:36.768018 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 19795
I20260809 02:06:36.776264 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 19976
I20260809 02:06:36.784235 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 20120
2026-08-09T02:06:36Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (60173 ms)
[----------] 30 tests from SecurityITest (254779 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786240941968412-1795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:06:36 dist-test-slave-ss4n krb5kdc[20321](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:06:36 dist-test-slave-ss4n krb5kdc[20321](info): set up 2 sockets
Aug 09 02:06:36 dist-test-slave-ss4n krb5kdc[20321](info): commencing operation
krb5kdc: starting...
W20260809 02:06:37.900410 1795 mini_kdc.cc:121] Time spent starting KDC: real 1.091s user 0.005s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786240941968412-1795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786240941968412-1795-0/krb5kdc/test-user.keytab.
Aug 09 02:06:37 dist-test-slave-ss4n krb5kdc[20321](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241197, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:06:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:06:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.192.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.192.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.192.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab.
Entry for principal kudu/127.1.192.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab.
I20260809 02:06:38.045783 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786240941968412-1795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786240941968412-1795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786240941968412-1795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.192.254:43507
--webserver_interface=127.1.192.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab
--principal=kudu/127.1.192.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:35867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.192.254:43507
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:06:38.165285 20334 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:06:38.165624 20334 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:06:38.165671 20334 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:06:38.170059 20334 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:06:38.170138 20334 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:06:38.170158 20334 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:06:38.170176 20334 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:06:38.170192 20334 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:06:38.176016 20334 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:35867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786240941968412-1795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786240941968412-1795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.192.254:43507
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab
--principal=kudu/127.1.192.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.192.254:43507
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.192.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20334
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786240941968412-1795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:06:38.177199 20334 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:06:38.178628 20334 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:06:38.187637 20341 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:06:38.187767 20334 server_base.cc:1034] running on GCE node
W20260809 02:06:38.187700 20339 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:06:38.188011 20343 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:06:38.188558 20334 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:06:38.189780 20334 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:06:38.190950 20334 hybrid_clock.cc:648] HybridClock initialized: now 1786241198190939 us; error 25 us; skew 500 ppm
Aug 09 02:06:38 dist-test-slave-ss4n krb5kdc[20321](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241198, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:06:38.193907 20334 init.cc:377] Logged in from keytab as kudu/127.1.192.254@KRBTEST.COM (short username kudu)
I20260809 02:06:38.194866 20334 webserver.cc:460] Webserver started at http://127.1.192.254:38699/ using document root <none> and password file <none>
I20260809 02:06:38.195487 20334 fs_manager.cc:359] Metadata directory not provided
I20260809 02:06:38.195583 20334 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:06:38.195829 20334 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:06:38.198097 20334 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786240941968412-1795-0/minicluster-data/master-0/data/instance:
uuid: "4edc900eb9dd41fda75bd375f8c8aadd"
format_stamp: "Formatted at 2026-08-09 02:06:38 on dist-test-slave-ss4n"
server_key: "5cac76ce1905cf33fdcf01ea93815034"
server_key_iv: "f575f4b4eacf4e2377caa81160ade63a"
server_key_version: "encryptionkey@0"
I20260809 02:06:38.198695 20334 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786240941968412-1795-0/minicluster-data/master-0/wal/instance:
uuid: "4edc900eb9dd41fda75bd375f8c8aadd"
format_stamp: "Formatted at 2026-08-09 02:06:38 on dist-test-slave-ss4n"
server_key: "5cac76ce1905cf33fdcf01ea93815034"
server_key_iv: "f575f4b4eacf4e2377caa81160ade63a"
server_key_version: "encryptionkey@0"
I20260809 02:06:38.202535 20334 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260809 02:06:38.204945 20350 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:06:38.206231 20334 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 02:06:38.206374 20334 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786240941968412-1795-0/minicluster-data/master-0/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786240941968412-1795-0/minicluster-data/master-0/wal
uuid: "4edc900eb9dd41fda75bd375f8c8aadd"
format_stamp: "Formatted at 2026-08-09 02:06:38 on dist-test-slave-ss4n"
server_key: "5cac76ce1905cf33fdcf01ea93815034"
server_key_iv: "f575f4b4eacf4e2377caa81160ade63a"
server_key_version: "encryptionkey@0"
I20260809 02:06:38.206490 20334 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786240941968412-1795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786240941968412-1795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786240941968412-1795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:06:38.231320 20334 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:06:38.234814 20334 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:06:38.235041 20334 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:06:38.242775 20334 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.254:43507
I20260809 02:06:38.242802 20408 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.254:43507 every 8 connection(s)
I20260809 02:06:38.243798 20334 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
I20260809 02:06:38.246675 20409 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:06:38.252486 20409 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4edc900eb9dd41fda75bd375f8c8aadd: Bootstrap starting.
I20260809 02:06:38.254115 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 20334
I20260809 02:06:38.254261 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786240941968412-1795-0/minicluster-data/master-0/wal/instance
I20260809 02:06:38.254554 1795 external_mini_cluster.cc:1428] Setting key 76865ce4332fe519d7e52bc0b9ab7a1e
I20260809 02:06:38.255636 20409 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4edc900eb9dd41fda75bd375f8c8aadd: Neither blocks nor log segments found. Creating new log.
I20260809 02:06:38.256544 20409 log.cc:824] T 00000000000000000000000000000000 P 4edc900eb9dd41fda75bd375f8c8aadd: Log is configured to *not* fsync() on all Append() calls
I20260809 02:06:38.258608 20409 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4edc900eb9dd41fda75bd375f8c8aadd: No bootstrap required, opened a new log
I20260809 02:06:38.263248 20409 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4edc900eb9dd41fda75bd375f8c8aadd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4edc900eb9dd41fda75bd375f8c8aadd" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 43507 } }
I20260809 02:06:38.263610 20409 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4edc900eb9dd41fda75bd375f8c8aadd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:06:38.263696 20409 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4edc900eb9dd41fda75bd375f8c8aadd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4edc900eb9dd41fda75bd375f8c8aadd, State: Initialized, Role: FOLLOWER
I20260809 02:06:38.264267 20409 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4edc900eb9dd41fda75bd375f8c8aadd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4edc900eb9dd41fda75bd375f8c8aadd" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 43507 } }
I20260809 02:06:38.264453 20409 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4edc900eb9dd41fda75bd375f8c8aadd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:06:38.264525 20409 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4edc900eb9dd41fda75bd375f8c8aadd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:06:38.264624 20409 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4edc900eb9dd41fda75bd375f8c8aadd [term 0 FOLLOWER]: Advancing to term 1
Aug 09 02:06:38 dist-test-slave-ss4n krb5kdc[20321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241197, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.192.254@KRBTEST.COM
I20260809 02:06:38.265758 20409 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4edc900eb9dd41fda75bd375f8c8aadd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4edc900eb9dd41fda75bd375f8c8aadd" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 43507 } }
I20260809 02:06:38.266158 20409 leader_election.cc:302] T 00000000000000000000000000000000 P 4edc900eb9dd41fda75bd375f8c8aadd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4edc900eb9dd41fda75bd375f8c8aadd; no voters:
I20260809 02:06:38.266541 20409 leader_election.cc:288] T 00000000000000000000000000000000 P 4edc900eb9dd41fda75bd375f8c8aadd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:06:38.266697 20414 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4edc900eb9dd41fda75bd375f8c8aadd [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:06:38.267031 20414 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4edc900eb9dd41fda75bd375f8c8aadd [term 1 LEADER]: Becoming Leader. State: Replica: 4edc900eb9dd41fda75bd375f8c8aadd, State: Running, Role: LEADER
I20260809 02:06:38.267463 20414 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4edc900eb9dd41fda75bd375f8c8aadd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4edc900eb9dd41fda75bd375f8c8aadd" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 43507 } }
I20260809 02:06:38.267998 20409 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4edc900eb9dd41fda75bd375f8c8aadd [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:06:38.268833 20415 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4edc900eb9dd41fda75bd375f8c8aadd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4edc900eb9dd41fda75bd375f8c8aadd. Latest consensus state: current_term: 1 leader_uuid: "4edc900eb9dd41fda75bd375f8c8aadd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4edc900eb9dd41fda75bd375f8c8aadd" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 43507 } } }
I20260809 02:06:38.269126 20415 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4edc900eb9dd41fda75bd375f8c8aadd [sys.catalog]: This master's current role is: LEADER
I20260809 02:06:38.269292 20416 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4edc900eb9dd41fda75bd375f8c8aadd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4edc900eb9dd41fda75bd375f8c8aadd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4edc900eb9dd41fda75bd375f8c8aadd" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 43507 } } }
I20260809 02:06:38.269413 20416 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4edc900eb9dd41fda75bd375f8c8aadd [sys.catalog]: This master's current role is: LEADER
I20260809 02:06:38.269788 20423 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:06:38.272353 20412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:38.255861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43892 (local address 127.1.192.254:43507)
0809 02:06:38.256345 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0809 02:06:38.256360 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 02:06:38.256399 (+ 39us) server_negotiation.cc:407] Connection header received
0809 02:06:38.257186 (+ 787us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:06:38.257207 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:06:38.257571 (+ 364us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:06:38.257985 (+ 414us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:06:38.258866 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:38.260354 (+ 1488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:38.261283 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:38.262187 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:38.262367 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:38.265906 (+ 3539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:06:38.265912 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:06:38.265931 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:06:38.265964 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:06:38.268323 (+ 2359us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:06:38.269008 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:06:38.269016 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:06:38.269024 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:06:38.269196 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:06:38.269628 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:06:38.269632 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:06:38.269634 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:06:38.270219 (+ 585us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:06:38.270438 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0809 02:06:38.270937 (+ 499us) server_negotiation.cc:299] Negotiation successful
0809 02:06:38.271288 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329,"thread_start_us":147,"threads_started":1}
I20260809 02:06:38.273933 20423 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:06:38.280918 20423 catalog_manager.cc:1347] Generated new cluster ID: 980a95ffb8744d0b92fcc2907e5d373d
I20260809 02:06:38.281014 20423 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:06:38.297145 20423 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:06:38.298300 20423 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:06:38.302475 20423 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4edc900eb9dd41fda75bd375f8c8aadd: Generated new TSK 0
I20260809 02:06:38.303320 20423 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.192.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.192.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.192.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.193.keytab.
Entry for principal kudu/127.1.192.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.193.keytab.
I20260809 02:06:38.345083 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786240941968412-1795-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786240941968412-1795-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786240941968412-1795-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786240941968412-1795-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.192.193:0
--local_ip_for_outbound_sockets=127.1.192.193
--webserver_interface=127.1.192.193
--webserver_port=0
--tserver_master_addrs=127.1.192.254:43507
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.193.keytab
--principal=kudu/127.1.192.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:35867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:06:38.475382 20435 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:06:38.475661 20435 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:06:38.475706 20435 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:06:38.479656 20435 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:06:38.479712 20435 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:06:38.479768 20435 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.192.193
I20260809 02:06:38.484951 20435 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:35867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786240941968412-1795-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786240941968412-1795-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.193.keytab
--principal=kudu/127.1.192.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.192.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786240941968412-1795-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.192.193
--webserver_port=0
--tserver_master_addrs=127.1.192.254:43507
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20435
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.192.193
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786240941968412-1795-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:06:38.485994 20435 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:06:38.487196 20435 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:06:38.497799 20441 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:06:38.497799 20444 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:06:38.497834 20442 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:06:38.498266 20435 server_base.cc:1034] running on GCE node
I20260809 02:06:38.498718 20435 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:06:38.499450 20435 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:06:38.500654 20435 hybrid_clock.cc:648] HybridClock initialized: now 1786241198500637 us; error 28 us; skew 500 ppm
Aug 09 02:06:38 dist-test-slave-ss4n krb5kdc[20321](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241198, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:06:38.503931 20435 init.cc:377] Logged in from keytab as kudu/127.1.192.193@KRBTEST.COM (short username kudu)
I20260809 02:06:38.504899 20435 webserver.cc:460] Webserver started at http://127.1.192.193:40045/ using document root <none> and password file <none>
I20260809 02:06:38.505573 20435 fs_manager.cc:359] Metadata directory not provided
I20260809 02:06:38.505672 20435 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:06:38.505923 20435 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:06:38.508143 20435 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786240941968412-1795-0/minicluster-data/ts-0/data/instance:
uuid: "c7768e48d27f4a099a9feb6581985635"
format_stamp: "Formatted at 2026-08-09 02:06:38 on dist-test-slave-ss4n"
server_key: "1353317dad4ca23cf326f4e6af4139f5"
server_key_iv: "a090837d9028c2db2e6fbc3543051c43"
server_key_version: "encryptionkey@0"
I20260809 02:06:38.508811 20435 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786240941968412-1795-0/minicluster-data/ts-0/wal/instance:
uuid: "c7768e48d27f4a099a9feb6581985635"
format_stamp: "Formatted at 2026-08-09 02:06:38 on dist-test-slave-ss4n"
server_key: "1353317dad4ca23cf326f4e6af4139f5"
server_key_iv: "a090837d9028c2db2e6fbc3543051c43"
server_key_version: "encryptionkey@0"
I20260809 02:06:38.513078 20435 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.004s
I20260809 02:06:38.515821 20451 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:06:38.517154 20435 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 02:06:38.517299 20435 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786240941968412-1795-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786240941968412-1795-0/minicluster-data/ts-0/wal
uuid: "c7768e48d27f4a099a9feb6581985635"
format_stamp: "Formatted at 2026-08-09 02:06:38 on dist-test-slave-ss4n"
server_key: "1353317dad4ca23cf326f4e6af4139f5"
server_key_iv: "a090837d9028c2db2e6fbc3543051c43"
server_key_version: "encryptionkey@0"
I20260809 02:06:38.517424 20435 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786240941968412-1795-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786240941968412-1795-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786240941968412-1795-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:06:38.536209 20435 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:06:38.539750 20435 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:06:38.539966 20435 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:06:38.540592 20435 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:06:38.541433 20435 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:06:38.541540 20435 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:06:38.541627 20435 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:06:38.541672 20435 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:06:38.550864 20435 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.193:34969
I20260809 02:06:38.550889 20571 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.193:34969 every 8 connection(s)
I20260809 02:06:38.551790 20435 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786240941968412-1795-0/minicluster-data/ts-0/data/info.pb
I20260809 02:06:38.553821 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 20435
I20260809 02:06:38.553972 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786240941968412-1795-0/minicluster-data/ts-0/wal/instance
I20260809 02:06:38.554314 1795 external_mini_cluster.cc:1428] Setting key 39791b5787668816d90cdecc856b13df
Aug 09 02:06:38 dist-test-slave-ss4n krb5kdc[20321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241198, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.193@KRBTEST.COM for kudu/127.1.192.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.192.194@KRBTEST.COM; defaulting to no policy
I20260809 02:06:38.569300 20412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:38.553722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.193:60487 (local address 127.1.192.254:43507)
0809 02:06:38.553906 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0809 02:06:38.553913 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 02:06:38.554860 (+ 947us) server_negotiation.cc:407] Connection header received
0809 02:06:38.556260 (+ 1400us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:06:38.556263 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:06:38.556310 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:06:38.556383 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:06:38.558056 (+ 1673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:38.559046 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:38.560437 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:38.560971 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:38.561124 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:38.564394 (+ 3270us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:06:38.564401 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:06:38.564407 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:06:38.564444 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:06:38.566550 (+ 2106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:06:38.567173 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:06:38.567176 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:06:38.567178 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:06:38.567271 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:06:38.567803 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:06:38.567805 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:06:38.567806 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:06:38.568083 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:06:38.568197 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0809 02:06:38.568928 (+ 731us) server_negotiation.cc:299] Negotiation successful
0809 02:06:38.569090 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.192.194@KRBTEST.COM" created.
I20260809 02:06:38.570559 20574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:38.554186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:43507 (local address 127.1.192.193:60487)
0809 02:06:38.554681 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0809 02:06:38.554739 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0809 02:06:38.556034 (+ 1295us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:06:38.556576 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:06:38.556587 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:06:38.557118 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:06:38.557869 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:06:38.557892 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:06:38.559230 (+ 1338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:06:38.559233 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:06:38.560287 (+ 1054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:06:38.560298 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:06:38.561228 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:06:38.561236 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:06:38.561404 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:38.562190 (+ 786us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:06:38.562208 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:06:38.564164 (+ 1956us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:06:38.566716 (+ 2552us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:06:38.566723 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:06:38.566739 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:06:38.567047 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:06:38.567458 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:06:38.567465 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:06:38.567471 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:06:38.567698 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:06:38.568253 (+ 555us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:06:38.568260 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:06:38.568568 (+ 308us) client_negotiation.cc:769] Sending connection context
0809 02:06:38.568882 (+ 314us) client_negotiation.cc:240] Negotiation successful
0809 02:06:38.569406 (+ 524us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":146,"threads_started":1}
I20260809 02:06:38.571980 20572 heartbeater.cc:347] Connected to a master server at 127.1.192.254:43507
I20260809 02:06:38.572285 20572 heartbeater.cc:464] Registering TS with master...
I20260809 02:06:38.572793 20572 heartbeater.cc:511] Master 127.1.192.254:43507 requested a full tablet report, sending...
I20260809 02:06:38.574412 20371 ts_manager.cc:194] Registered new tserver with Master: c7768e48d27f4a099a9feb6581985635 (127.1.192.193:34969)
I20260809 02:06:38.575628 20371 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.192.193@KRBTEST.COM'} at 127.1.192.193:60487
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.192.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.194.keytab.
Entry for principal kudu/127.1.192.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.194.keytab.
I20260809 02:06:38.590461 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786240941968412-1795-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786240941968412-1795-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786240941968412-1795-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786240941968412-1795-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.192.194:0
--local_ip_for_outbound_sockets=127.1.192.194
--webserver_interface=127.1.192.194
--webserver_port=0
--tserver_master_addrs=127.1.192.254:43507
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.194.keytab
--principal=kudu/127.1.192.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:35867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:06:38.708361 20577 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:06:38.708654 20577 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:06:38.708700 20577 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:06:38.712972 20577 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:06:38.713048 20577 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:06:38.713143 20577 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.192.194
I20260809 02:06:38.719808 20577 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:35867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786240941968412-1795-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786240941968412-1795-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.194.keytab
--principal=kudu/127.1.192.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.192.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786240941968412-1795-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.192.194
--webserver_port=0
--tserver_master_addrs=127.1.192.254:43507
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20577
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.192.194
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786240941968412-1795-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:06:38.721153 20577 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:06:38.722674 20577 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:06:38.733330 20585 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:06:38.733279 20582 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:06:38.733330 20583 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:06:38.733304 20577 server_base.cc:1034] running on GCE node
I20260809 02:06:38.734064 20577 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:06:38.734789 20577 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:06:38.735983 20577 hybrid_clock.cc:648] HybridClock initialized: now 1786241198735953 us; error 57 us; skew 500 ppm
Aug 09 02:06:38 dist-test-slave-ss4n krb5kdc[20321](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241198, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:06:38.739229 20577 init.cc:377] Logged in from keytab as kudu/127.1.192.194@KRBTEST.COM (short username kudu)
I20260809 02:06:38.740147 20577 webserver.cc:460] Webserver started at http://127.1.192.194:34947/ using document root <none> and password file <none>
I20260809 02:06:38.740734 20577 fs_manager.cc:359] Metadata directory not provided
I20260809 02:06:38.740798 20577 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:06:38.740976 20577 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:06:38.743861 20577 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786240941968412-1795-0/minicluster-data/ts-1/data/instance:
uuid: "a1492c81d3134110a9ccbb92855ab231"
format_stamp: "Formatted at 2026-08-09 02:06:38 on dist-test-slave-ss4n"
server_key: "0879f68187a7b320c6b694c6aecc288e"
server_key_iv: "8838a9876ffc970279a278707927659f"
server_key_version: "encryptionkey@0"
I20260809 02:06:38.744554 20577 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786240941968412-1795-0/minicluster-data/ts-1/wal/instance:
uuid: "a1492c81d3134110a9ccbb92855ab231"
format_stamp: "Formatted at 2026-08-09 02:06:38 on dist-test-slave-ss4n"
server_key: "0879f68187a7b320c6b694c6aecc288e"
server_key_iv: "8838a9876ffc970279a278707927659f"
server_key_version: "encryptionkey@0"
I20260809 02:06:38.749107 20577 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 02:06:38.751977 20592 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:06:38.753599 20577 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260809 02:06:38.753743 20577 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786240941968412-1795-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786240941968412-1795-0/minicluster-data/ts-1/wal
uuid: "a1492c81d3134110a9ccbb92855ab231"
format_stamp: "Formatted at 2026-08-09 02:06:38 on dist-test-slave-ss4n"
server_key: "0879f68187a7b320c6b694c6aecc288e"
server_key_iv: "8838a9876ffc970279a278707927659f"
server_key_version: "encryptionkey@0"
I20260809 02:06:38.753877 20577 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786240941968412-1795-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786240941968412-1795-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786240941968412-1795-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:06:38.781869 20577 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:06:38.785732 20577 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:06:38.785972 20577 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:06:38.786664 20577 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:06:38.787604 20577 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:06:38.787657 20577 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:06:38.787724 20577 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:06:38.787767 20577 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:06:38.797648 20577 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.194:34491
I20260809 02:06:38.797765 20714 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.194:34491 every 8 connection(s)
I20260809 02:06:38.798672 20577 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786240941968412-1795-0/minicluster-data/ts-1/data/info.pb
Aug 09 02:06:38 dist-test-slave-ss4n krb5kdc[20321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241198, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.194@KRBTEST.COM for kudu/127.1.192.254@KRBTEST.COM
I20260809 02:06:38.808729 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 20577
I20260809 02:06:38.808864 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786240941968412-1795-0/minicluster-data/ts-1/wal/instance
I20260809 02:06:38.809271 1795 external_mini_cluster.cc:1428] Setting key 2253dcabad8d990aec9cbeec84e602a4
I20260809 02:06:38.814077 20412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:38.800622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.194:58923 (local address 127.1.192.254:43507)
0809 02:06:38.800772 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0809 02:06:38.800776 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:06:38.801673 (+ 897us) server_negotiation.cc:407] Connection header received
0809 02:06:38.803183 (+ 1510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:06:38.803187 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:06:38.803249 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:06:38.803346 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:06:38.804547 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:38.805083 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:38.806303 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:38.806526 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:38.806639 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:38.809428 (+ 2789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:06:38.809431 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:06:38.809434 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:06:38.809460 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:06:38.811300 (+ 1840us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:06:38.812037 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:06:38.812040 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:06:38.812041 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:06:38.812123 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:06:38.812657 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:06:38.812659 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:06:38.812661 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:06:38.812857 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:06:38.812950 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 02:06:38.813691 (+ 741us) server_negotiation.cc:299] Negotiation successful
0809 02:06:38.813851 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 02:06:38.814934 20718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:38.801023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:43507 (local address 127.1.192.194:58923)
0809 02:06:38.801514 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0809 02:06:38.801568 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 02:06:38.802955 (+ 1387us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:06:38.803520 (+ 565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:06:38.803526 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:06:38.803897 (+ 371us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:06:38.804398 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:06:38.804411 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:06:38.805225 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:06:38.805229 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:06:38.806174 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:06:38.806185 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:06:38.806686 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:06:38.806690 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:06:38.806854 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:38.807450 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:06:38.807466 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:06:38.809208 (+ 1742us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:06:38.811424 (+ 2216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:06:38.811431 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:06:38.811451 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:06:38.811917 (+ 466us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:06:38.812240 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:06:38.812246 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:06:38.812251 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:06:38.812458 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:06:38.812972 (+ 514us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:06:38.812982 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:06:38.813369 (+ 387us) client_negotiation.cc:769] Sending connection context
0809 02:06:38.813631 (+ 262us) client_negotiation.cc:240] Negotiation successful
0809 02:06:38.813934 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":125,"threads_started":1}
I20260809 02:06:38.816206 20716 heartbeater.cc:347] Connected to a master server at 127.1.192.254:43507
I20260809 02:06:38.816498 20716 heartbeater.cc:464] Registering TS with master...
I20260809 02:06:38.817067 20716 heartbeater.cc:511] Master 127.1.192.254:43507 requested a full tablet report, sending...
I20260809 02:06:38.818254 20371 ts_manager.cc:194] Registered new tserver with Master: a1492c81d3134110a9ccbb92855ab231 (127.1.192.194:34491)
I20260809 02:06:38.819083 20371 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.192.194@KRBTEST.COM'} at 127.1.192.194:58923
WARNING: no policy specified for kudu/127.1.192.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.192.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.192.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.195.keytab.
Entry for principal kudu/127.1.192.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.195.keytab.
I20260809 02:06:38.840973 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786240941968412-1795-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786240941968412-1795-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786240941968412-1795-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786240941968412-1795-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.192.195:0
--local_ip_for_outbound_sockets=127.1.192.195
--webserver_interface=127.1.192.195
--webserver_port=0
--tserver_master_addrs=127.1.192.254:43507
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.195.keytab
--principal=kudu/127.1.192.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:35867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:06:38.958762 20721 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:06:38.959055 20721 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:06:38.959100 20721 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:06:38.963330 20721 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:06:38.963413 20721 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:06:38.963588 20721 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.192.195
I20260809 02:06:38.969405 20721 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:35867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786240941968412-1795-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786240941968412-1795-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.195.keytab
--principal=kudu/127.1.192.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.192.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786240941968412-1795-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.192.195
--webserver_port=0
--tserver_master_addrs=127.1.192.254:43507
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20721
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.192.195
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786240941968412-1795-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:06:38.970697 20721 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:06:38.971752 20721 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:06:38.981737 20729 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:06:38.981702 20727 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:06:38.981755 20726 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:06:38.982147 20721 server_base.cc:1034] running on GCE node
I20260809 02:06:38.982579 20721 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:06:38.983250 20721 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:06:38.984444 20721 hybrid_clock.cc:648] HybridClock initialized: now 1786241198984406 us; error 51 us; skew 500 ppm
Aug 09 02:06:38 dist-test-slave-ss4n krb5kdc[20321](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241198, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:06:38.987516 20721 init.cc:377] Logged in from keytab as kudu/127.1.192.195@KRBTEST.COM (short username kudu)
I20260809 02:06:38.988561 20721 webserver.cc:460] Webserver started at http://127.1.192.195:37177/ using document root <none> and password file <none>
I20260809 02:06:38.989142 20721 fs_manager.cc:359] Metadata directory not provided
I20260809 02:06:38.989233 20721 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:06:38.989461 20721 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:06:38.991627 20721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786240941968412-1795-0/minicluster-data/ts-2/data/instance:
uuid: "bf54e48e9bab49c79ea95160d7098c25"
format_stamp: "Formatted at 2026-08-09 02:06:38 on dist-test-slave-ss4n"
server_key: "33c1afb9dfe299916fdd5975738f1e12"
server_key_iv: "2aaa75d6330fb01e83af2520f1d317f5"
server_key_version: "encryptionkey@0"
I20260809 02:06:38.992216 20721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786240941968412-1795-0/minicluster-data/ts-2/wal/instance:
uuid: "bf54e48e9bab49c79ea95160d7098c25"
format_stamp: "Formatted at 2026-08-09 02:06:38 on dist-test-slave-ss4n"
server_key: "33c1afb9dfe299916fdd5975738f1e12"
server_key_iv: "2aaa75d6330fb01e83af2520f1d317f5"
server_key_version: "encryptionkey@0"
I20260809 02:06:38.996268 20721 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.002s
I20260809 02:06:38.998926 20736 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:06:39.000159 20721 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 02:06:39.000304 20721 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786240941968412-1795-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786240941968412-1795-0/minicluster-data/ts-2/wal
uuid: "bf54e48e9bab49c79ea95160d7098c25"
format_stamp: "Formatted at 2026-08-09 02:06:38 on dist-test-slave-ss4n"
server_key: "33c1afb9dfe299916fdd5975738f1e12"
server_key_iv: "2aaa75d6330fb01e83af2520f1d317f5"
server_key_version: "encryptionkey@0"
I20260809 02:06:39.000659 20721 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786240941968412-1795-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786240941968412-1795-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786240941968412-1795-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:06:39.022234 20721 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:06:39.025749 20721 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:06:39.025992 20721 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:06:39.026641 20721 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:06:39.027491 20721 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:06:39.027561 20721 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:06:39.027638 20721 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:06:39.027683 20721 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:06:39.036855 20721 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.195:45559
I20260809 02:06:39.036875 20859 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.195:45559 every 8 connection(s)
I20260809 02:06:39.037967 20721 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786240941968412-1795-0/minicluster-data/ts-2/data/info.pb
I20260809 02:06:39.039074 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 20721
I20260809 02:06:39.039184 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786240941968412-1795-0/minicluster-data/ts-2/wal/instance
I20260809 02:06:39.039528 1795 external_mini_cluster.cc:1428] Setting key 19eb8593f5c8b3bb45f7735f59a53438
Aug 09 02:06:39 dist-test-slave-ss4n krb5kdc[20321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241198, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.195@KRBTEST.COM for kudu/127.1.192.254@KRBTEST.COM
I20260809 02:06:39.053296 20412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:39.040096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.195:52359 (local address 127.1.192.254:43507)
0809 02:06:39.040259 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0809 02:06:39.040264 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:06:39.041243 (+ 979us) server_negotiation.cc:407] Connection header received
0809 02:06:39.042822 (+ 1579us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:06:39.042826 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:06:39.042886 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:06:39.042984 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:06:39.044252 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:39.044838 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:39.046159 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:39.046406 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:39.046532 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:39.049324 (+ 2792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:06:39.049327 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:06:39.049330 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:06:39.049358 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:06:39.050965 (+ 1607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:06:39.051481 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:06:39.051484 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:06:39.051485 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:06:39.051563 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:06:39.051954 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:06:39.051956 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:06:39.051957 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:06:39.052153 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:06:39.052247 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0809 02:06:39.052973 (+ 726us) server_negotiation.cc:299] Negotiation successful
0809 02:06:39.053107 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 02:06:39.054879 20863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:39.040521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:43507 (local address 127.1.192.195:52359)
0809 02:06:39.041067 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0809 02:06:39.041135 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0809 02:06:39.042594 (+ 1459us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:06:39.043165 (+ 571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:06:39.043172 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:06:39.043584 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:06:39.044097 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:06:39.044110 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:06:39.044983 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:06:39.044988 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:06:39.046017 (+ 1029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:06:39.046028 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:06:39.046582 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:06:39.046586 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:06:39.046786 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:39.047406 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:06:39.047423 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:06:39.049132 (+ 1709us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:06:39.051092 (+ 1960us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:06:39.051097 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:06:39.051111 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:06:39.051387 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:06:39.051690 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:06:39.051693 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:06:39.051695 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:06:39.051858 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:06:39.052292 (+ 434us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:06:39.052299 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:06:39.052648 (+ 349us) client_negotiation.cc:769] Sending connection context
0809 02:06:39.053616 (+ 968us) client_negotiation.cc:240] Negotiation successful
0809 02:06:39.053959 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":140,"threads_started":1}
I20260809 02:06:39.055994 20860 heartbeater.cc:347] Connected to a master server at 127.1.192.254:43507
I20260809 02:06:39.056272 20860 heartbeater.cc:464] Registering TS with master...
I20260809 02:06:39.056825 20860 heartbeater.cc:511] Master 127.1.192.254:43507 requested a full tablet report, sending...
I20260809 02:06:39.057983 20371 ts_manager.cc:194] Registered new tserver with Master: bf54e48e9bab49c79ea95160d7098c25 (127.1.192.195:45559)
I20260809 02:06:39.058677 20371 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.192.195@KRBTEST.COM'} at 127.1.192.195:52359
I20260809 02:06:39.066430 1795 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 02:06:39.077203 20412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:39.068849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43906 (local address 127.1.192.254:43507)
0809 02:06:39.069051 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0809 02:06:39.069056 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:06:39.069266 (+ 210us) server_negotiation.cc:407] Connection header received
0809 02:06:39.069505 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:06:39.069510 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:06:39.069565 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:06:39.069667 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:06:39.070483 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:39.071051 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:39.071968 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:39.072205 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:39.072318 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:39.073838 (+ 1520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:06:39.073842 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:06:39.073845 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:06:39.073873 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:06:39.075431 (+ 1558us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:06:39.075873 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:06:39.075877 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:06:39.075879 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:06:39.075967 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:06:39.076245 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:06:39.076250 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:06:39.076252 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:06:39.076492 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:06:39.076642 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0809 02:06:39.076876 (+ 234us) server_negotiation.cc:299] Negotiation successful
0809 02:06:39.077018 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260809 02:06:39.080606 20371 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43906:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 02:06:39.084102 20371 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:06:39.099020 20872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:39.093228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.195:45559 (local address 127.0.0.1:56284)
0809 02:06:39.093797 (+ 569us) negotiation.cc:107] Waiting for socket to connect
0809 02:06:39.093861 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0809 02:06:39.094013 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:06:39.094776 (+ 763us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:06:39.094781 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:06:39.094807 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:06:39.094947 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:06:39.094953 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:06:39.096143 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:06:39.096148 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:06:39.097370 (+ 1222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:06:39.097380 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:06:39.098473 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:06:39.098484 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:06:39.098593 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:39.098629 (+ 36us) client_negotiation.cc:769] Sending connection context
0809 02:06:39.098745 (+ 116us) client_negotiation.cc:240] Negotiation successful
0809 02:06:39.098845 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":377,"spinlock_wait_cycles":4864,"thread_start_us":105,"threads_started":1}
I20260809 02:06:39.099324 20873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:39.093406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56284 (local address 127.1.192.195:45559)
0809 02:06:39.093985 (+ 579us) server_negotiation.cc:206] Beginning negotiation
0809 02:06:39.093999 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 02:06:39.094027 (+ 28us) server_negotiation.cc:407] Connection header received
0809 02:06:39.094234 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:06:39.094240 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:06:39.094495 (+ 255us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:06:39.094660 (+ 165us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:06:39.095083 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:39.096019 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:39.097706 (+ 1687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:39.098344 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:39.098472 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:39.098630 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0809 02:06:39.099034 (+ 404us) server_negotiation.cc:299] Negotiation successful
0809 02:06:39.099164 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":460,"thread_start_us":218,"threads_started":1}
I20260809 02:06:39.100603 20875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:39.094159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.193:34969 (local address 127.0.0.1:50438)
0809 02:06:39.094760 (+ 601us) negotiation.cc:107] Waiting for socket to connect
0809 02:06:39.094774 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 02:06:39.094873 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:06:39.095641 (+ 768us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:06:39.095644 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:06:39.095663 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:06:39.095794 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:06:39.095804 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:06:39.097664 (+ 1860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:06:39.097668 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:06:39.099035 (+ 1367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:06:39.099045 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:06:39.100203 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:06:39.100207 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:06:39.100305 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:39.100331 (+ 26us) client_negotiation.cc:769] Sending connection context
0809 02:06:39.100386 (+ 55us) client_negotiation.cc:240] Negotiation successful
0809 02:06:39.100459 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":538,"spinlock_wait_cycles":1280,"thread_start_us":95,"threads_started":1}
I20260809 02:06:39.100797 20876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:39.093227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.194:34491 (local address 127.0.0.1:52256)
0809 02:06:39.094430 (+ 1203us) negotiation.cc:107] Waiting for socket to connect
0809 02:06:39.094446 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 02:06:39.094542 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:06:39.095521 (+ 979us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:06:39.095525 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:06:39.095545 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:06:39.095639 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:06:39.095645 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:06:39.097244 (+ 1599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:06:39.097248 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:06:39.098830 (+ 1582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:06:39.098842 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:06:39.100308 (+ 1466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:06:39.100312 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:06:39.100404 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:39.100466 (+ 62us) client_negotiation.cc:769] Sending connection context
0809 02:06:39.100541 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 02:06:39.100639 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1122,"mutex_wait_us":625,"thread_start_us":89,"threads_started":1}
I20260809 02:06:39.101358 20877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:39.094599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50438 (local address 127.1.192.193:34969)
0809 02:06:39.095129 (+ 530us) server_negotiation.cc:206] Beginning negotiation
0809 02:06:39.095135 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:06:39.095155 (+ 20us) server_negotiation.cc:407] Connection header received
0809 02:06:39.095221 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:06:39.095225 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:06:39.095464 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:06:39.095975 (+ 511us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:06:39.096114 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:39.097523 (+ 1409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:39.099183 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:39.100043 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:39.100917 (+ 874us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:39.101070 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0809 02:06:39.101122 (+ 52us) server_negotiation.cc:299] Negotiation successful
0809 02:06:39.101212 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":427,"thread_start_us":76,"threads_started":1}
I20260809 02:06:39.101363 20874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:39.093462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52256 (local address 127.1.192.194:34491)
0809 02:06:39.095025 (+ 1563us) server_negotiation.cc:206] Beginning negotiation
0809 02:06:39.095038 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 02:06:39.095064 (+ 26us) server_negotiation.cc:407] Connection header received
0809 02:06:39.095144 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:06:39.095150 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:06:39.095350 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:06:39.095535 (+ 185us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:06:39.095748 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:39.097082 (+ 1334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:39.098968 (+ 1886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:39.099926 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:39.100056 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:39.100220 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0809 02:06:39.101086 (+ 866us) server_negotiation.cc:299] Negotiation successful
0809 02:06:39.101196 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1451,"thread_start_us":135,"threads_started":1}
I20260809 02:06:39.102756 20787 tablet_service.cc:1467] Processing CreateTablet for tablet 8ad8eea3e23d4b55b76e7bbfdca3aaf1 (DEFAULT_TABLE table=test-table [id=f2bd24ea90314c9d89e7063ce4557c66]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:06:39.103806 20787 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ad8eea3e23d4b55b76e7bbfdca3aaf1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:06:39.104049 20504 tablet_service.cc:1467] Processing CreateTablet for tablet 8ad8eea3e23d4b55b76e7bbfdca3aaf1 (DEFAULT_TABLE table=test-table [id=f2bd24ea90314c9d89e7063ce4557c66]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:06:39.104041 20645 tablet_service.cc:1467] Processing CreateTablet for tablet 8ad8eea3e23d4b55b76e7bbfdca3aaf1 (DEFAULT_TABLE table=test-table [id=f2bd24ea90314c9d89e7063ce4557c66]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:06:39.105070 20504 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ad8eea3e23d4b55b76e7bbfdca3aaf1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:06:39.105072 20645 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ad8eea3e23d4b55b76e7bbfdca3aaf1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:06:39.110733 20878 tablet_bootstrap.cc:492] T 8ad8eea3e23d4b55b76e7bbfdca3aaf1 P bf54e48e9bab49c79ea95160d7098c25: Bootstrap starting.
I20260809 02:06:39.112241 20879 tablet_bootstrap.cc:492] T 8ad8eea3e23d4b55b76e7bbfdca3aaf1 P a1492c81d3134110a9ccbb92855ab231: Bootstrap starting.
I20260809 02:06:39.112896 20880 tablet_bootstrap.cc:492] T 8ad8eea3e23d4b55b76e7bbfdca3aaf1 P c7768e48d27f4a099a9feb6581985635: Bootstrap starting.
I20260809 02:06:39.113440 20878 tablet_bootstrap.cc:654] T 8ad8eea3e23d4b55b76e7bbfdca3aaf1 P bf54e48e9bab49c79ea95160d7098c25: Neither blocks nor log segments found. Creating new log.
I20260809 02:06:39.114631 20880 tablet_bootstrap.cc:654] T 8ad8eea3e23d4b55b76e7bbfdca3aaf1 P c7768e48d27f4a099a9feb6581985635: Neither blocks nor log segments found. Creating new log.
I20260809 02:06:39.114898 20878 log.cc:824] T 8ad8eea3e23d4b55b76e7bbfdca3aaf1 P bf54e48e9bab49c79ea95160d7098c25: Log is configured to *not* fsync() on all Append() calls
I20260809 02:06:39.115054 20879 tablet_bootstrap.cc:654] T 8ad8eea3e23d4b55b76e7bbfdca3aaf1 P a1492c81d3134110a9ccbb92855ab231: Neither blocks nor log segments found. Creating new log.
I20260809 02:06:39.115383 20880 log.cc:824] T 8ad8eea3e23d4b55b76e7bbfdca3aaf1 P c7768e48d27f4a099a9feb6581985635: Log is configured to *not* fsync() on all Append() calls
I20260809 02:06:39.115921 20879 log.cc:824] T 8ad8eea3e23d4b55b76e7bbfdca3aaf1 P a1492c81d3134110a9ccbb92855ab231: Log is configured to *not* fsync() on all Append() calls
I20260809 02:06:39.117368 20880 tablet_bootstrap.cc:492] T 8ad8eea3e23d4b55b76e7bbfdca3aaf1 P c7768e48d27f4a099a9feb6581985635: No bootstrap required, opened a new log
I20260809 02:06:39.117611 20880 ts_tablet_manager.cc:1397] T 8ad8eea3e23d4b55b76e7bbfdca3aaf1 P c7768e48d27f4a099a9feb6581985635: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.003s
I20260809 02:06:39.117728 20878 tablet_bootstrap.cc:492] T 8ad8eea3e23d4b55b76e7bbfdca3aaf1 P bf54e48e9bab49c79ea95160d7098c25: No bootstrap required, opened a new log
I20260809 02:06:39.117928 20878 ts_tablet_manager.cc:1397] T 8ad8eea3e23d4b55b76e7bbfdca3aaf1 P bf54e48e9bab49c79ea95160d7098c25: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260809 02:06:39.117985 20879 tablet_bootstrap.cc:492] T 8ad8eea3e23d4b55b76e7bbfdca3aaf1 P a1492c81d3134110a9ccbb92855ab231: No bootstrap required, opened a new log
I20260809 02:06:39.118155 20879 ts_tablet_manager.cc:1397] T 8ad8eea3e23d4b55b76e7bbfdca3aaf1 P a1492c81d3134110a9ccbb92855ab231: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260809 02:06:39.121286 20880 raft_consensus.cc:348] T 8ad8eea3e23d4b55b76e7bbfdca3aaf1 P c7768e48d27f4a099a9feb6581985635 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf54e48e9bab49c79ea95160d7098c25" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 45559 } } peers { permanent_uuid: "a1492c81d3134110a9ccbb92855ab231" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 34491 } } peers { permanent_uuid: "c7768e48d27f4a099a9feb6581985635" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 34969 } }
I20260809 02:06:39.121559 20880 raft_consensus.cc:374] T 8ad8eea3e23d4b55b76e7bbfdca3aaf1 P c7768e48d27f4a099a9feb6581985635 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:06:39.121614 20880 raft_consensus.cc:729] T 8ad8eea3e23d4b55b76e7bbfdca3aaf1 P c7768e48d27f4a099a9feb6581985635 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7768e48d27f4a099a9feb6581985635, State: Initialized, Role: FOLLOWER
I20260809 02:06:39.122062 20880 consensus_queue.cc:260] T 8ad8eea3e23d4b55b76e7bbfdca3aaf1 P c7768e48d27f4a099a9feb6581985635 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf54e48e9bab49c79ea95160d7098c25" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 45559 } } peers { permanent_uuid: "a1492c81d3134110a9ccbb92855ab231" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 34491 } } peers { permanent_uuid: "c7768e48d27f4a099a9feb6581985635" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 34969 } }
I20260809 02:06:39.122272 20878 raft_consensus.cc:348] T 8ad8eea3e23d4b55b76e7bbfdca3aaf1 P bf54e48e9bab49c79ea95160d7098c25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf54e48e9bab49c79ea95160d7098c25" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 45559 } } peers { permanent_uuid: "a1492c81d3134110a9ccbb92855ab231" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 34491 } } peers { permanent_uuid: "c7768e48d27f4a099a9feb6581985635" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 34969 } }
I20260809 02:06:39.122262 20879 raft_consensus.cc:348] T 8ad8eea3e23d4b55b76e7bbfdca3aaf1 P a1492c81d3134110a9ccbb92855ab231 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf54e48e9bab49c79ea95160d7098c25" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 45559 } } peers { permanent_uuid: "a1492c81d3134110a9ccbb92855ab231" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 34491 } } peers { permanent_uuid: "c7768e48d27f4a099a9feb6581985635" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 34969 } }
I20260809 02:06:39.122540 20879 raft_consensus.cc:374] T 8ad8eea3e23d4b55b76e7bbfdca3aaf1 P a1492c81d3134110a9ccbb92855ab231 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:06:39.122540 20878 raft_consensus.cc:374] T 8ad8eea3e23d4b55b76e7bbfdca3aaf1 P bf54e48e9bab49c79ea95160d7098c25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:06:39.122599 20879 raft_consensus.cc:729] T 8ad8eea3e23d4b55b76e7bbfdca3aaf1 P a1492c81d3134110a9ccbb92855ab231 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1492c81d3134110a9ccbb92855ab231, State: Initialized, Role: FOLLOWER
I20260809 02:06:39.122608 20878 raft_consensus.cc:729] T 8ad8eea3e23d4b55b76e7bbfdca3aaf1 P bf54e48e9bab49c79ea95160d7098c25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf54e48e9bab49c79ea95160d7098c25, State: Initialized, Role: FOLLOWER
I20260809 02:06:39.123086 20879 consensus_queue.cc:260] T 8ad8eea3e23d4b55b76e7bbfdca3aaf1 P a1492c81d3134110a9ccbb92855ab231 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf54e48e9bab49c79ea95160d7098c25" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 45559 } } peers { permanent_uuid: "a1492c81d3134110a9ccbb92855ab231" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 34491 } } peers { permanent_uuid: "c7768e48d27f4a099a9feb6581985635" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 34969 } }
I20260809 02:06:39.123086 20878 consensus_queue.cc:260] T 8ad8eea3e23d4b55b76e7bbfdca3aaf1 P bf54e48e9bab49c79ea95160d7098c25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf54e48e9bab49c79ea95160d7098c25" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 45559 } } peers { permanent_uuid: "a1492c81d3134110a9ccbb92855ab231" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 34491 } } peers { permanent_uuid: "c7768e48d27f4a099a9feb6581985635" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 34969 } }
I20260809 02:06:39.123270 20880 ts_tablet_manager.cc:1428] T 8ad8eea3e23d4b55b76e7bbfdca3aaf1 P c7768e48d27f4a099a9feb6581985635: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260809 02:06:39.123646 20572 heartbeater.cc:503] Master 127.1.192.254:43507 was elected leader, sending a full tablet report...
I20260809 02:06:39.124075 20878 ts_tablet_manager.cc:1428] T 8ad8eea3e23d4b55b76e7bbfdca3aaf1 P bf54e48e9bab49c79ea95160d7098c25: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260809 02:06:39.124075 20879 ts_tablet_manager.cc:1428] T 8ad8eea3e23d4b55b76e7bbfdca3aaf1 P a1492c81d3134110a9ccbb92855ab231: Time spent starting tablet: real 0.006s user 0.002s sys 0.004s
I20260809 02:06:39.124439 20716 heartbeater.cc:503] Master 127.1.192.254:43507 was elected leader, sending a full tablet report...
I20260809 02:06:39.124691 20860 heartbeater.cc:503] Master 127.1.192.254:43507 was elected leader, sending a full tablet report...
W20260809 02:06:39.289028 20862 tablet.cc:2369] T 8ad8eea3e23d4b55b76e7bbfdca3aaf1 P bf54e48e9bab49c79ea95160d7098c25: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 02:06:39.299935 20717 tablet.cc:2369] T 8ad8eea3e23d4b55b76e7bbfdca3aaf1 P a1492c81d3134110a9ccbb92855ab231: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 02:06:39.303187 20573 tablet.cc:2369] T 8ad8eea3e23d4b55b76e7bbfdca3aaf1 P c7768e48d27f4a099a9feb6581985635: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 02:06:39.407135 20884 raft_consensus.cc:482] T 8ad8eea3e23d4b55b76e7bbfdca3aaf1 P c7768e48d27f4a099a9feb6581985635 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:06:39.407384 20884 raft_consensus.cc:504] T 8ad8eea3e23d4b55b76e7bbfdca3aaf1 P c7768e48d27f4a099a9feb6581985635 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf54e48e9bab49c79ea95160d7098c25" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 45559 } } peers { permanent_uuid: "a1492c81d3134110a9ccbb92855ab231" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 34491 } } peers { permanent_uuid: "c7768e48d27f4a099a9feb6581985635" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 34969 } }
I20260809 02:06:39.408654 20884 leader_election.cc:288] T 8ad8eea3e23d4b55b76e7bbfdca3aaf1 P c7768e48d27f4a099a9feb6581985635 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bf54e48e9bab49c79ea95160d7098c25 (127.1.192.195:45559), a1492c81d3134110a9ccbb92855ab231 (127.1.192.194:34491)
I20260809 02:06:39.412837 20873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:39.408993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.193:53895 (local address 127.1.192.195:45559)
0809 02:06:39.409279 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0809 02:06:39.409286 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 02:06:39.409309 (+ 23us) server_negotiation.cc:407] Connection header received
0809 02:06:39.409509 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:06:39.409516 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:06:39.409575 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:06:39.409668 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:06:39.410007 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:39.410602 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:39.411813 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:39.412254 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:39.412349 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:39.412389 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0809 02:06:39.412613 (+ 224us) server_negotiation.cc:299] Negotiation successful
0809 02:06:39.412705 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157}
I20260809 02:06:39.412871 20887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:39.408895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.195:45559 (local address 127.1.192.193:53895)
0809 02:06:39.409206 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0809 02:06:39.409225 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 02:06:39.409344 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:06:39.409698 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:06:39.409701 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:06:39.409723 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:06:39.409900 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:06:39.409910 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:06:39.410742 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:06:39.410745 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:06:39.411692 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:06:39.411703 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:06:39.412376 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:06:39.412381 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:06:39.412484 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:39.412507 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 02:06:39.412582 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 02:06:39.412693 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":226,"mutex_wait_us":29,"thread_start_us":125,"threads_started":1}
I20260809 02:06:39.412904 20874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:39.409085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.193:58279 (local address 127.1.192.194:34491)
0809 02:06:39.409306 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0809 02:06:39.409310 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:06:39.409573 (+ 263us) server_negotiation.cc:407] Connection header received
0809 02:06:39.409711 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:06:39.409714 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:06:39.409766 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:06:39.409851 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:06:39.410172 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:39.410747 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:39.411941 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:39.412359 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:39.412478 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:39.412524 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0809 02:06:39.412718 (+ 194us) server_negotiation.cc:299] Negotiation successful
0809 02:06:39.412790 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120}
I20260809 02:06:39.412878 20888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:39.408981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.194:34491 (local address 127.1.192.193:58279)
0809 02:06:39.409462 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0809 02:06:39.409476 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 02:06:39.409592 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:06:39.409867 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:06:39.409869 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:06:39.409883 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:06:39.410027 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:06:39.410034 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:06:39.410865 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:06:39.410868 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:06:39.411821 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:06:39.411831 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:06:39.412481 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:06:39.412483 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:06:39.412583 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:39.412608 (+ 25us) client_negotiation.cc:769] Sending connection context
0809 02:06:39.412684 (+ 76us) client_negotiation.cc:240] Negotiation successful
0809 02:06:39.412744 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":104,"threads_started":1}
I20260809 02:06:39.413599 20667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ad8eea3e23d4b55b76e7bbfdca3aaf1" candidate_uuid: "c7768e48d27f4a099a9feb6581985635" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1492c81d3134110a9ccbb92855ab231" is_pre_election: true
I20260809 02:06:39.413604 20811 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ad8eea3e23d4b55b76e7bbfdca3aaf1" candidate_uuid: "c7768e48d27f4a099a9feb6581985635" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf54e48e9bab49c79ea95160d7098c25" is_pre_election: true
I20260809 02:06:39.413895 20667 raft_consensus.cc:2454] T 8ad8eea3e23d4b55b76e7bbfdca3aaf1 P a1492c81d3134110a9ccbb92855ab231 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7768e48d27f4a099a9feb6581985635 in term 0.
I20260809 02:06:39.413900 20811 raft_consensus.cc:2454] T 8ad8eea3e23d4b55b76e7bbfdca3aaf1 P bf54e48e9bab49c79ea95160d7098c25 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7768e48d27f4a099a9feb6581985635 in term 0.
I20260809 02:06:39.414390 20454 leader_election.cc:302] T 8ad8eea3e23d4b55b76e7bbfdca3aaf1 P c7768e48d27f4a099a9feb6581985635 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a1492c81d3134110a9ccbb92855ab231, c7768e48d27f4a099a9feb6581985635; no voters:
I20260809 02:06:39.414662 20884 raft_consensus.cc:2781] T 8ad8eea3e23d4b55b76e7bbfdca3aaf1 P c7768e48d27f4a099a9feb6581985635 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:06:39.414753 20884 raft_consensus.cc:482] T 8ad8eea3e23d4b55b76e7bbfdca3aaf1 P c7768e48d27f4a099a9feb6581985635 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:06:39.414798 20884 raft_consensus.cc:3037] T 8ad8eea3e23d4b55b76e7bbfdca3aaf1 P c7768e48d27f4a099a9feb6581985635 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:06:39.415887 20884 raft_consensus.cc:504] T 8ad8eea3e23d4b55b76e7bbfdca3aaf1 P c7768e48d27f4a099a9feb6581985635 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf54e48e9bab49c79ea95160d7098c25" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 45559 } } peers { permanent_uuid: "a1492c81d3134110a9ccbb92855ab231" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 34491 } } peers { permanent_uuid: "c7768e48d27f4a099a9feb6581985635" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 34969 } }
I20260809 02:06:39.416337 20884 leader_election.cc:288] T 8ad8eea3e23d4b55b76e7bbfdca3aaf1 P c7768e48d27f4a099a9feb6581985635 [CANDIDATE]: Term 1 election: Requested vote from peers bf54e48e9bab49c79ea95160d7098c25 (127.1.192.195:45559), a1492c81d3134110a9ccbb92855ab231 (127.1.192.194:34491)
I20260809 02:06:39.416702 20811 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ad8eea3e23d4b55b76e7bbfdca3aaf1" candidate_uuid: "c7768e48d27f4a099a9feb6581985635" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf54e48e9bab49c79ea95160d7098c25"
I20260809 02:06:39.416817 20811 raft_consensus.cc:3037] T 8ad8eea3e23d4b55b76e7bbfdca3aaf1 P bf54e48e9bab49c79ea95160d7098c25 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:06:39.416790 20667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ad8eea3e23d4b55b76e7bbfdca3aaf1" candidate_uuid: "c7768e48d27f4a099a9feb6581985635" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1492c81d3134110a9ccbb92855ab231"
I20260809 02:06:39.416922 20667 raft_consensus.cc:3037] T 8ad8eea3e23d4b55b76e7bbfdca3aaf1 P a1492c81d3134110a9ccbb92855ab231 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:06:39.417948 20811 raft_consensus.cc:2454] T 8ad8eea3e23d4b55b76e7bbfdca3aaf1 P bf54e48e9bab49c79ea95160d7098c25 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7768e48d27f4a099a9feb6581985635 in term 1.
I20260809 02:06:39.418130 20667 raft_consensus.cc:2454] T 8ad8eea3e23d4b55b76e7bbfdca3aaf1 P a1492c81d3134110a9ccbb92855ab231 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7768e48d27f4a099a9feb6581985635 in term 1.
I20260809 02:06:39.418551 20453 leader_election.cc:302] T 8ad8eea3e23d4b55b76e7bbfdca3aaf1 P c7768e48d27f4a099a9feb6581985635 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bf54e48e9bab49c79ea95160d7098c25, c7768e48d27f4a099a9feb6581985635; no voters:
I20260809 02:06:39.418820 20884 raft_consensus.cc:2781] T 8ad8eea3e23d4b55b76e7bbfdca3aaf1 P c7768e48d27f4a099a9feb6581985635 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:06:39.419060 20884 raft_consensus.cc:686] T 8ad8eea3e23d4b55b76e7bbfdca3aaf1 P c7768e48d27f4a099a9feb6581985635 [term 1 LEADER]: Becoming Leader. State: Replica: c7768e48d27f4a099a9feb6581985635, State: Running, Role: LEADER
I20260809 02:06:39.419397 20884 consensus_queue.cc:237] T 8ad8eea3e23d4b55b76e7bbfdca3aaf1 P c7768e48d27f4a099a9feb6581985635 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf54e48e9bab49c79ea95160d7098c25" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 45559 } } peers { permanent_uuid: "a1492c81d3134110a9ccbb92855ab231" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 34491 } } peers { permanent_uuid: "c7768e48d27f4a099a9feb6581985635" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 34969 } }
I20260809 02:06:39.422909 20370 catalog_manager.cc:5495] T 8ad8eea3e23d4b55b76e7bbfdca3aaf1 P c7768e48d27f4a099a9feb6581985635 reported cstate change: term changed from 0 to 1, leader changed from <none> to c7768e48d27f4a099a9feb6581985635 (127.1.192.193). New cstate: current_term: 1 leader_uuid: "c7768e48d27f4a099a9feb6581985635" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf54e48e9bab49c79ea95160d7098c25" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 45559 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a1492c81d3134110a9ccbb92855ab231" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 34491 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7768e48d27f4a099a9feb6581985635" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 34969 } health_report { overall_health: HEALTHY } } }
I20260809 02:06:39.465224 20877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:39.461701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50454 (local address 127.1.192.193:34969)
0809 02:06:39.461824 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0809 02:06:39.461827 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:06:39.461843 (+ 16us) server_negotiation.cc:407] Connection header received
0809 02:06:39.461920 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:06:39.461922 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:06:39.461977 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:06:39.462071 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:06:39.462395 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:39.462919 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:39.463804 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:39.464061 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:39.464158 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:39.464454 (+ 296us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:06:39.464892 (+ 438us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:06:39.464983 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 02:06:39.465059 (+ 76us) server_negotiation.cc:299] Negotiation successful
0809 02:06:39.465110 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 02:06:39.471587 20667 raft_consensus.cc:1261] T 8ad8eea3e23d4b55b76e7bbfdca3aaf1 P a1492c81d3134110a9ccbb92855ab231 [term 1 FOLLOWER]: Refusing update from remote peer c7768e48d27f4a099a9feb6581985635: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:06:39.471587 20811 raft_consensus.cc:1261] T 8ad8eea3e23d4b55b76e7bbfdca3aaf1 P bf54e48e9bab49c79ea95160d7098c25 [term 1 FOLLOWER]: Refusing update from remote peer c7768e48d27f4a099a9feb6581985635: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:06:39.472414 20889 consensus_queue.cc:1035] T 8ad8eea3e23d4b55b76e7bbfdca3aaf1 P c7768e48d27f4a099a9feb6581985635 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf54e48e9bab49c79ea95160d7098c25" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 45559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:06:39.472679 20884 consensus_queue.cc:1035] T 8ad8eea3e23d4b55b76e7bbfdca3aaf1 P c7768e48d27f4a099a9feb6581985635 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a1492c81d3134110a9ccbb92855ab231" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 34491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:06:39.480639 20893 mvcc.cc:205] Tried to move back new op lower bound from 7316443953028149248 to 7316443952824238080. Current Snapshot: MvccSnapshot[applied={T|T < 7316443953028149248}]
I20260809 02:06:39.480844 20894 mvcc.cc:205] Tried to move back new op lower bound from 7316443953028149248 to 7316443952824238080. Current Snapshot: MvccSnapshot[applied={T|T < 7316443953028149248}]
I20260809 02:06:39.490120 20370 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43906:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:06:39.490286 20370 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43906:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:06:39.492688 20370 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4edc900eb9dd41fda75bd375f8c8aadd: Sending DeleteTablet for 3 replicas of tablet 8ad8eea3e23d4b55b76e7bbfdca3aaf1
I20260809 02:06:39.493602 20645 tablet_service.cc:1514] Processing DeleteTablet for tablet 8ad8eea3e23d4b55b76e7bbfdca3aaf1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:06:39 UTC) from {username='kudu', principal='kudu/127.1.192.254@KRBTEST.COM'} at 127.0.0.1:52256
I20260809 02:06:39.493692 20787 tablet_service.cc:1514] Processing DeleteTablet for tablet 8ad8eea3e23d4b55b76e7bbfdca3aaf1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:06:39 UTC) from {username='kudu', principal='kudu/127.1.192.254@KRBTEST.COM'} at 127.0.0.1:56284
I20260809 02:06:39.493799 20504 tablet_service.cc:1514] Processing DeleteTablet for tablet 8ad8eea3e23d4b55b76e7bbfdca3aaf1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:06:39 UTC) from {username='kudu', principal='kudu/127.1.192.254@KRBTEST.COM'} at 127.0.0.1:50438
I20260809 02:06:39.494261 20903 tablet_replica.cc:330] T 8ad8eea3e23d4b55b76e7bbfdca3aaf1 P bf54e48e9bab49c79ea95160d7098c25: stopping tablet replica
I20260809 02:06:39.494494 20904 tablet_replica.cc:330] T 8ad8eea3e23d4b55b76e7bbfdca3aaf1 P c7768e48d27f4a099a9feb6581985635: stopping tablet replica
I20260809 02:06:39.494287 20902 tablet_replica.cc:330] T 8ad8eea3e23d4b55b76e7bbfdca3aaf1 P a1492c81d3134110a9ccbb92855ab231: stopping tablet replica
I20260809 02:06:39.494604 20903 raft_consensus.cc:2229] T 8ad8eea3e23d4b55b76e7bbfdca3aaf1 P bf54e48e9bab49c79ea95160d7098c25 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:06:39.494827 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 20435
I20260809 02:06:39.495954 20903 raft_consensus.cc:2258] T 8ad8eea3e23d4b55b76e7bbfdca3aaf1 P bf54e48e9bab49c79ea95160d7098c25 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:06:39.498245 20902 raft_consensus.cc:2229] T 8ad8eea3e23d4b55b76e7bbfdca3aaf1 P a1492c81d3134110a9ccbb92855ab231 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:06:39.498628 20902 raft_consensus.cc:2258] T 8ad8eea3e23d4b55b76e7bbfdca3aaf1 P a1492c81d3134110a9ccbb92855ab231 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:06:39.499338 20903 ts_tablet_manager.cc:1905] T 8ad8eea3e23d4b55b76e7bbfdca3aaf1 P bf54e48e9bab49c79ea95160d7098c25: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:06:39.500109 20902 ts_tablet_manager.cc:1905] T 8ad8eea3e23d4b55b76e7bbfdca3aaf1 P a1492c81d3134110a9ccbb92855ab231: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:06:39.502352 20903 ts_tablet_manager.cc:1918] T 8ad8eea3e23d4b55b76e7bbfdca3aaf1 P bf54e48e9bab49c79ea95160d7098c25: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:06:39.502462 20903 log.cc:1196] T 8ad8eea3e23d4b55b76e7bbfdca3aaf1 P bf54e48e9bab49c79ea95160d7098c25: Deleting WAL directory at /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786240941968412-1795-0/minicluster-data/ts-2/wal/wals/8ad8eea3e23d4b55b76e7bbfdca3aaf1
I20260809 02:06:39.502805 20903 ts_tablet_manager.cc:1939] T 8ad8eea3e23d4b55b76e7bbfdca3aaf1 P bf54e48e9bab49c79ea95160d7098c25: Deleting consensus metadata
I20260809 02:06:39.503073 20902 ts_tablet_manager.cc:1918] T 8ad8eea3e23d4b55b76e7bbfdca3aaf1 P a1492c81d3134110a9ccbb92855ab231: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:06:39.503175 20902 log.cc:1196] T 8ad8eea3e23d4b55b76e7bbfdca3aaf1 P a1492c81d3134110a9ccbb92855ab231: Deleting WAL directory at /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786240941968412-1795-0/minicluster-data/ts-1/wal/wals/8ad8eea3e23d4b55b76e7bbfdca3aaf1
I20260809 02:06:39.503480 20902 ts_tablet_manager.cc:1939] T 8ad8eea3e23d4b55b76e7bbfdca3aaf1 P a1492c81d3134110a9ccbb92855ab231: Deleting consensus metadata
W20260809 02:06:39.504101 20353 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.192.193:34969 (error 108)
I20260809 02:06:39.504627 20354 catalog_manager.cc:4841] TS a1492c81d3134110a9ccbb92855ab231 (127.1.192.194:34491): tablet 8ad8eea3e23d4b55b76e7bbfdca3aaf1 (table test-table [id=f2bd24ea90314c9d89e7063ce4557c66]) successfully deleted
I20260809 02:06:39.504669 20353 catalog_manager.cc:4841] TS bf54e48e9bab49c79ea95160d7098c25 (127.1.192.195:45559): tablet 8ad8eea3e23d4b55b76e7bbfdca3aaf1 (table test-table [id=f2bd24ea90314c9d89e7063ce4557c66]) successfully deleted
I20260809 02:06:39.505264 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 20577
I20260809 02:06:39.505453 20876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:39.505070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.193:34969 (local address 127.0.0.1:50460)
0809 02:06:39.505262 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0809 02:06:39.505351 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.193:34969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
W20260809 02:06:39.506768 20353 catalog_manager.cc:4584] TS c7768e48d27f4a099a9feb6581985635 (127.1.192.193:34969): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8ad8eea3e23d4b55b76e7bbfdca3aaf1: Network error: Client connection negotiation failed: client connection to 127.1.192.193:34969: connect: Connection refused (error 111)
I20260809 02:06:39.514592 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 20721
I20260809 02:06:39.521996 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 20334
2026-08-09T02:06:39Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2736 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-09T02:06:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:06:39Z Disabled control of system clock
I20260809 02:06:39.555819 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786240941968412-1795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786240941968412-1795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786240941968412-1795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.192.254:33353
--webserver_interface=127.1.192.254
--webserver_port=0
--builtin_ntp_servers=127.1.192.212:46761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.192.254:33353
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 02:06:39.667585 20909 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:06:39.667898 20909 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:06:39.667936 20909 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:06:39.672062 20909 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:06:39.672133 20909 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:06:39.672149 20909 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:06:39.672166 20909 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:06:39.672183 20909 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:06:39.677165 20909 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:46761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786240941968412-1795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786240941968412-1795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.192.254:33353
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.192.254:33353
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.192.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20909
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786240941968412-1795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:06:39.678377 20909 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:06:39.679733 20909 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:06:39.688373 20915 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:06:39.688385 20914 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:06:39.688373 20917 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:06:39.688935 20909 server_base.cc:1034] running on GCE node
I20260809 02:06:39.689464 20909 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:06:39.690575 20909 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:06:39.691735 20909 hybrid_clock.cc:648] HybridClock initialized: now 1786241199691705 us; error 44 us; skew 500 ppm
I20260809 02:06:39.693677 20909 webserver.cc:460] Webserver started at http://127.1.192.254:32903/ using document root <none> and password file <none>
I20260809 02:06:39.694255 20909 fs_manager.cc:359] Metadata directory not provided
I20260809 02:06:39.694320 20909 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:06:39.694535 20909 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:06:39.696688 20909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786240941968412-1795-0/minicluster-data/master-0/data/instance:
uuid: "08e8346ff27c476884ad09902069a35a"
format_stamp: "Formatted at 2026-08-09 02:06:39 on dist-test-slave-ss4n"
server_key: "a2a83825e297cf1002f36528fe8356a1"
server_key_iv: "f23f2e77655fe9342549491fc61dc0a2"
server_key_version: "encryptionkey@0"
I20260809 02:06:39.697288 20909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786240941968412-1795-0/minicluster-data/master-0/wal/instance:
uuid: "08e8346ff27c476884ad09902069a35a"
format_stamp: "Formatted at 2026-08-09 02:06:39 on dist-test-slave-ss4n"
server_key: "a2a83825e297cf1002f36528fe8356a1"
server_key_iv: "f23f2e77655fe9342549491fc61dc0a2"
server_key_version: "encryptionkey@0"
I20260809 02:06:39.701133 20909 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.000s
I20260809 02:06:39.703822 20925 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:06:39.705165 20909 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 02:06:39.705317 20909 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786240941968412-1795-0/minicluster-data/master-0/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786240941968412-1795-0/minicluster-data/master-0/wal
uuid: "08e8346ff27c476884ad09902069a35a"
format_stamp: "Formatted at 2026-08-09 02:06:39 on dist-test-slave-ss4n"
server_key: "a2a83825e297cf1002f36528fe8356a1"
server_key_iv: "f23f2e77655fe9342549491fc61dc0a2"
server_key_version: "encryptionkey@0"
I20260809 02:06:39.705439 20909 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786240941968412-1795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786240941968412-1795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786240941968412-1795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:06:39.729928 20909 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:06:39.730695 20909 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:06:39.730896 20909 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:06:39.738368 20909 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.254:33353
I20260809 02:06:39.738389 20981 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.254:33353 every 8 connection(s)
I20260809 02:06:39.739481 20909 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
I20260809 02:06:39.742512 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 20909
I20260809 02:06:39.742655 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786240941968412-1795-0/minicluster-data/master-0/wal/instance
I20260809 02:06:39.742842 20982 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:06:39.742980 1795 external_mini_cluster.cc:1428] Setting key 8882120fc8bde53a28d94f02d4a97c8b
I20260809 02:06:39.751065 20982 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 08e8346ff27c476884ad09902069a35a: Bootstrap starting.
I20260809 02:06:39.753539 20985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:39.744435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44332 (local address 127.1.192.254:33353)
0809 02:06:39.745134 (+ 699us) server_negotiation.cc:206] Beginning negotiation
0809 02:06:39.745153 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0809 02:06:39.745190 (+ 37us) server_negotiation.cc:407] Connection header received
0809 02:06:39.746051 (+ 861us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:06:39.746075 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:06:39.746593 (+ 518us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:06:39.747013 (+ 420us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:06:39.747574 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:39.749041 (+ 1467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:39.750015 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:39.750521 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:39.750702 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:39.751674 (+ 972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:06:39.751686 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:06:39.751700 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:06:39.752095 (+ 395us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:06:39.752166 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:06:39.752185 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:06:39.752297 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 02:06:39.752451 (+ 154us) server_negotiation.cc:299] Negotiation successful
0809 02:06:39.752760 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":171,"threads_started":1}
I20260809 02:06:39.753649 20982 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 08e8346ff27c476884ad09902069a35a: Neither blocks nor log segments found. Creating new log.
I20260809 02:06:39.754654 20982 log.cc:824] T 00000000000000000000000000000000 P 08e8346ff27c476884ad09902069a35a: Log is configured to *not* fsync() on all Append() calls
I20260809 02:06:39.756904 20982 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 08e8346ff27c476884ad09902069a35a: No bootstrap required, opened a new log
I20260809 02:06:39.761042 20982 raft_consensus.cc:348] T 00000000000000000000000000000000 P 08e8346ff27c476884ad09902069a35a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08e8346ff27c476884ad09902069a35a" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 33353 } }
I20260809 02:06:39.761281 20982 raft_consensus.cc:374] T 00000000000000000000000000000000 P 08e8346ff27c476884ad09902069a35a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:06:39.761322 20982 raft_consensus.cc:729] T 00000000000000000000000000000000 P 08e8346ff27c476884ad09902069a35a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08e8346ff27c476884ad09902069a35a, State: Initialized, Role: FOLLOWER
I20260809 02:06:39.761759 20982 consensus_queue.cc:260] T 00000000000000000000000000000000 P 08e8346ff27c476884ad09902069a35a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08e8346ff27c476884ad09902069a35a" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 33353 } }
I20260809 02:06:39.761889 20982 raft_consensus.cc:388] T 00000000000000000000000000000000 P 08e8346ff27c476884ad09902069a35a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:06:39.761947 20982 raft_consensus.cc:482] T 00000000000000000000000000000000 P 08e8346ff27c476884ad09902069a35a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:06:39.762028 20982 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 08e8346ff27c476884ad09902069a35a [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:06:39.763232 20982 raft_consensus.cc:504] T 00000000000000000000000000000000 P 08e8346ff27c476884ad09902069a35a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08e8346ff27c476884ad09902069a35a" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 33353 } }
I20260809 02:06:39.763631 20982 leader_election.cc:302] T 00000000000000000000000000000000 P 08e8346ff27c476884ad09902069a35a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 08e8346ff27c476884ad09902069a35a; no voters:
I20260809 02:06:39.763924 20982 leader_election.cc:288] T 00000000000000000000000000000000 P 08e8346ff27c476884ad09902069a35a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:06:39.764039 20987 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 08e8346ff27c476884ad09902069a35a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:06:39.764285 20987 raft_consensus.cc:686] T 00000000000000000000000000000000 P 08e8346ff27c476884ad09902069a35a [term 1 LEADER]: Becoming Leader. State: Replica: 08e8346ff27c476884ad09902069a35a, State: Running, Role: LEADER
I20260809 02:06:39.764600 20987 consensus_queue.cc:237] T 00000000000000000000000000000000 P 08e8346ff27c476884ad09902069a35a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08e8346ff27c476884ad09902069a35a" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 33353 } }
I20260809 02:06:39.765240 20982 sys_catalog.cc:564] T 00000000000000000000000000000000 P 08e8346ff27c476884ad09902069a35a [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:06:39.766228 20989 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08e8346ff27c476884ad09902069a35a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 08e8346ff27c476884ad09902069a35a. Latest consensus state: current_term: 1 leader_uuid: "08e8346ff27c476884ad09902069a35a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08e8346ff27c476884ad09902069a35a" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 33353 } } }
I20260809 02:06:39.766391 20989 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08e8346ff27c476884ad09902069a35a [sys.catalog]: This master's current role is: LEADER
I20260809 02:06:39.766438 20988 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08e8346ff27c476884ad09902069a35a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "08e8346ff27c476884ad09902069a35a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08e8346ff27c476884ad09902069a35a" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 33353 } } }
I20260809 02:06:39.766544 20988 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08e8346ff27c476884ad09902069a35a [sys.catalog]: This master's current role is: LEADER
I20260809 02:06:39.766897 20996 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:06:39.770032 20996 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:06:39.775288 20996 catalog_manager.cc:1347] Generated new cluster ID: 142f7dd3d29442e58f22bb9c788ac500
I20260809 02:06:39.775364 20996 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:06:39.783428 20996 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:06:39.784310 20996 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:06:39.791754 20996 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 08e8346ff27c476884ad09902069a35a: Generated new TSK 0
I20260809 02:06:39.792484 20996 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:06:39.798557 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786240941968412-1795-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786240941968412-1795-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786240941968412-1795-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786240941968412-1795-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.192.193:0
--local_ip_for_outbound_sockets=127.1.192.193
--webserver_interface=127.1.192.193
--webserver_port=0
--tserver_master_addrs=127.1.192.254:33353
--builtin_ntp_servers=127.1.192.212:46761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 02:06:39.914170 21006 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:06:39.914563 21006 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:06:39.914650 21006 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:06:39.921514 21006 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:06:39.921609 21006 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:06:39.921713 21006 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.192.193
I20260809 02:06:39.927534 21006 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:46761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786240941968412-1795-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786240941968412-1795-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.192.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786240941968412-1795-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.192.193
--webserver_port=0
--tserver_master_addrs=127.1.192.254:33353
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21006
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.192.193
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786240941968412-1795-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:06:39.928700 21006 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:06:39.930202 21006 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:06:39.940002 21014 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:06:39.940002 21011 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:06:39.940007 21012 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:06:39.940236 21006 server_base.cc:1034] running on GCE node
I20260809 02:06:39.940802 21006 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:06:39.941466 21006 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:06:39.942713 21006 hybrid_clock.cc:648] HybridClock initialized: now 1786241199942698 us; error 35 us; skew 500 ppm
I20260809 02:06:39.944703 21006 webserver.cc:460] Webserver started at http://127.1.192.193:44527/ using document root <none> and password file <none>
I20260809 02:06:39.945313 21006 fs_manager.cc:359] Metadata directory not provided
I20260809 02:06:39.945402 21006 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:06:39.945675 21006 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:06:39.947758 21006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786240941968412-1795-0/minicluster-data/ts-0/data/instance:
uuid: "8679bc66961e4c98a9e970f36aa7e528"
format_stamp: "Formatted at 2026-08-09 02:06:39 on dist-test-slave-ss4n"
server_key: "a66e0bbeed0a9cd761760f601c3c7ef5"
server_key_iv: "c78b878138c41cf20fc1cb37d0ebfbc3"
server_key_version: "encryptionkey@0"
I20260809 02:06:39.948326 21006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786240941968412-1795-0/minicluster-data/ts-0/wal/instance:
uuid: "8679bc66961e4c98a9e970f36aa7e528"
format_stamp: "Formatted at 2026-08-09 02:06:39 on dist-test-slave-ss4n"
server_key: "a66e0bbeed0a9cd761760f601c3c7ef5"
server_key_iv: "c78b878138c41cf20fc1cb37d0ebfbc3"
server_key_version: "encryptionkey@0"
I20260809 02:06:39.952169 21006 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260809 02:06:39.954867 21020 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:06:39.956259 21006 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 02:06:39.956447 21006 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786240941968412-1795-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786240941968412-1795-0/minicluster-data/ts-0/wal
uuid: "8679bc66961e4c98a9e970f36aa7e528"
format_stamp: "Formatted at 2026-08-09 02:06:39 on dist-test-slave-ss4n"
server_key: "a66e0bbeed0a9cd761760f601c3c7ef5"
server_key_iv: "c78b878138c41cf20fc1cb37d0ebfbc3"
server_key_version: "encryptionkey@0"
I20260809 02:06:39.956591 21006 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786240941968412-1795-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786240941968412-1795-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786240941968412-1795-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:06:39.974291 21006 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:06:39.975100 21006 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:06:39.975342 21006 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:06:39.976030 21006 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:06:39.977149 21006 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:06:39.977232 21006 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:06:39.977311 21006 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:06:39.977366 21006 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:06:39.988155 21006 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.193:43115
I20260809 02:06:39.988227 21139 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.193:43115 every 8 connection(s)
I20260809 02:06:39.989264 21006 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786240941968412-1795-0/minicluster-data/ts-0/data/info.pb
I20260809 02:06:39.996446 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 21006
I20260809 02:06:39.996603 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786240941968412-1795-0/minicluster-data/ts-0/wal/instance
I20260809 02:06:39.996881 1795 external_mini_cluster.cc:1428] Setting key 8c442194c720b6fd4b5c254a361654df
I20260809 02:06:39.999241 20985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:39.991330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.193:43015 (local address 127.1.192.254:33353)
0809 02:06:39.991507 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0809 02:06:39.991511 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:06:39.992561 (+ 1050us) server_negotiation.cc:407] Connection header received
0809 02:06:39.994055 (+ 1494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:06:39.994059 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:06:39.994118 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:06:39.994225 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:06:39.995114 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:39.995658 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:39.996906 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:39.997198 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:39.997311 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:39.998728 (+ 1417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:06:39.998734 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:06:39.998739 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:06:39.998754 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:06:39.998781 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:06:39.998791 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:06:39.998861 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 02:06:39.999022 (+ 161us) server_negotiation.cc:299] Negotiation successful
0809 02:06:39.999089 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 02:06:39.999521 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786240941968412-1795-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786240941968412-1795-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786240941968412-1795-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786240941968412-1795-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.192.194:0
--local_ip_for_outbound_sockets=127.1.192.194
--webserver_interface=127.1.192.194
--webserver_port=0
--tserver_master_addrs=127.1.192.254:33353
--builtin_ntp_servers=127.1.192.212:46761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 02:06:40.000273 21142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:39.991750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:33353 (local address 127.1.192.193:43015)
0809 02:06:39.992401 (+ 651us) negotiation.cc:107] Waiting for socket to connect
0809 02:06:39.992442 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 02:06:39.993833 (+ 1391us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:06:39.994395 (+ 562us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:06:39.994401 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:06:39.994486 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:06:39.994966 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:06:39.994979 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:06:39.995785 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:06:39.995789 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:06:39.996782 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:06:39.996792 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:06:39.997361 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:06:39.997365 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:06:39.997625 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:39.998400 (+ 775us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 02:06:39.998402 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:06:39.998458 (+ 56us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 02:06:39.998535 (+ 77us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:06:39.998860 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:06:39.998874 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:06:39.998883 (+ 9us) client_negotiation.cc:769] Sending connection context
0809 02:06:39.999021 (+ 138us) client_negotiation.cc:240] Negotiation successful
0809 02:06:39.999318 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":126,"threads_started":1}
I20260809 02:06:40.001523 21140 heartbeater.cc:347] Connected to a master server at 127.1.192.254:33353
I20260809 02:06:40.001773 21140 heartbeater.cc:464] Registering TS with master...
I20260809 02:06:40.002297 21140 heartbeater.cc:511] Master 127.1.192.254:33353 requested a full tablet report, sending...
I20260809 02:06:40.003703 20945 ts_manager.cc:194] Registered new tserver with Master: 8679bc66961e4c98a9e970f36aa7e528 (127.1.192.193:43115)
I20260809 02:06:40.004814 20945 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.192.193:43015
W20260809 02:06:40.117336 21143 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:06:40.117725 21143 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:06:40.117808 21143 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:06:40.121883 21143 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:06:40.121971 21143 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:06:40.122067 21143 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.192.194
I20260809 02:06:40.127031 21143 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:46761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786240941968412-1795-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786240941968412-1795-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.192.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786240941968412-1795-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.192.194
--webserver_port=0
--tserver_master_addrs=127.1.192.254:33353
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21143
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.192.194
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786240941968412-1795-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:06:40.128139 21143 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:06:40.129545 21143 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:06:40.139215 21148 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:06:40.139196 21149 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:06:40.139408 21143 server_base.cc:1034] running on GCE node
W20260809 02:06:40.139513 21152 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:06:40.139935 21143 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:06:40.140584 21143 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:06:40.141767 21143 hybrid_clock.cc:648] HybridClock initialized: now 1786241200141751 us; error 27 us; skew 500 ppm
I20260809 02:06:40.143675 21143 webserver.cc:460] Webserver started at http://127.1.192.194:41455/ using document root <none> and password file <none>
I20260809 02:06:40.144253 21143 fs_manager.cc:359] Metadata directory not provided
I20260809 02:06:40.144343 21143 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:06:40.144567 21143 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:06:40.146773 21143 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786240941968412-1795-0/minicluster-data/ts-1/data/instance:
uuid: "074ba1ee21c7451fa67494da5b2ee170"
format_stamp: "Formatted at 2026-08-09 02:06:40 on dist-test-slave-ss4n"
server_key: "64625eaf027ac4abda66d4e64b045817"
server_key_iv: "434c70c346c4a641a0d520716f8a52b8"
server_key_version: "encryptionkey@0"
I20260809 02:06:40.147346 21143 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786240941968412-1795-0/minicluster-data/ts-1/wal/instance:
uuid: "074ba1ee21c7451fa67494da5b2ee170"
format_stamp: "Formatted at 2026-08-09 02:06:40 on dist-test-slave-ss4n"
server_key: "64625eaf027ac4abda66d4e64b045817"
server_key_iv: "434c70c346c4a641a0d520716f8a52b8"
server_key_version: "encryptionkey@0"
I20260809 02:06:40.151033 21143 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 02:06:40.153374 21159 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:06:40.154554 21143 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 02:06:40.154696 21143 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786240941968412-1795-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786240941968412-1795-0/minicluster-data/ts-1/wal
uuid: "074ba1ee21c7451fa67494da5b2ee170"
format_stamp: "Formatted at 2026-08-09 02:06:40 on dist-test-slave-ss4n"
server_key: "64625eaf027ac4abda66d4e64b045817"
server_key_iv: "434c70c346c4a641a0d520716f8a52b8"
server_key_version: "encryptionkey@0"
I20260809 02:06:40.154811 21143 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786240941968412-1795-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786240941968412-1795-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786240941968412-1795-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:06:40.172770 21143 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:06:40.173566 21143 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:06:40.173779 21143 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:06:40.174399 21143 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:06:40.175385 21143 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:06:40.175465 21143 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:06:40.175551 21143 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:06:40.175596 21143 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:06:40.184347 21143 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.194:33745
I20260809 02:06:40.184371 21282 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.194:33745 every 8 connection(s)
I20260809 02:06:40.185293 21143 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786240941968412-1795-0/minicluster-data/ts-1/data/info.pb
I20260809 02:06:40.187480 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 21143
I20260809 02:06:40.187598 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786240941968412-1795-0/minicluster-data/ts-1/wal/instance
I20260809 02:06:40.187899 1795 external_mini_cluster.cc:1428] Setting key 4e4874852850ee81f04cfecc612e723d
I20260809 02:06:40.189913 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786240941968412-1795-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786240941968412-1795-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786240941968412-1795-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786240941968412-1795-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.192.195:0
--local_ip_for_outbound_sockets=127.1.192.195
--webserver_interface=127.1.192.195
--webserver_port=0
--tserver_master_addrs=127.1.192.254:33353
--builtin_ntp_servers=127.1.192.212:46761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 02:06:40.195739 20985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:40.187147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.194:47743 (local address 127.1.192.254:33353)
0809 02:06:40.187297 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0809 02:06:40.187302 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:06:40.188517 (+ 1215us) server_negotiation.cc:407] Connection header received
0809 02:06:40.189973 (+ 1456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:06:40.189977 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:06:40.190036 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:06:40.190127 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:06:40.191069 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:40.191886 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:40.193519 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:40.193920 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:40.194049 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:40.195133 (+ 1084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:06:40.195149 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:06:40.195155 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:06:40.195175 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:06:40.195204 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:06:40.195218 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:06:40.195302 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 02:06:40.195448 (+ 146us) server_negotiation.cc:299] Negotiation successful
0809 02:06:40.195529 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 02:06:40.196523 21285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:40.187622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:33353 (local address 127.1.192.194:47743)
0809 02:06:40.188360 (+ 738us) negotiation.cc:107] Waiting for socket to connect
0809 02:06:40.188413 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 02:06:40.189760 (+ 1347us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:06:40.190319 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:06:40.190325 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:06:40.190407 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:06:40.190921 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:06:40.190934 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:06:40.192030 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:06:40.192035 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:06:40.193360 (+ 1325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:06:40.193371 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:06:40.194072 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:06:40.194075 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:06:40.194236 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:40.194866 (+ 630us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 02:06:40.194868 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:06:40.194909 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 02:06:40.194974 (+ 65us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:06:40.195316 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:06:40.195323 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:06:40.195327 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 02:06:40.195420 (+ 93us) client_negotiation.cc:240] Negotiation successful
0809 02:06:40.195645 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":138,"threads_started":1}
I20260809 02:06:40.197855 21283 heartbeater.cc:347] Connected to a master server at 127.1.192.254:33353
I20260809 02:06:40.198132 21283 heartbeater.cc:464] Registering TS with master...
I20260809 02:06:40.198833 21283 heartbeater.cc:511] Master 127.1.192.254:33353 requested a full tablet report, sending...
I20260809 02:06:40.200042 20945 ts_manager.cc:194] Registered new tserver with Master: 074ba1ee21c7451fa67494da5b2ee170 (127.1.192.194:33745)
I20260809 02:06:40.200692 20945 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.192.194:47743
W20260809 02:06:40.306365 21287 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:06:40.306653 21287 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:06:40.306728 21287 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:06:40.311106 21287 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:06:40.311173 21287 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:06:40.311254 21287 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.192.195
I20260809 02:06:40.316072 21287 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:46761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786240941968412-1795-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786240941968412-1795-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.192.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786240941968412-1795-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.192.195
--webserver_port=0
--tserver_master_addrs=127.1.192.254:33353
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21287
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.192.195
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786240941968412-1795-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:06:40.317206 21287 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:06:40.318663 21287 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:06:40.328619 21293 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:06:40.328620 21296 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:06:40.328622 21294 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:06:40.329107 21287 server_base.cc:1034] running on GCE node
I20260809 02:06:40.329595 21287 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:06:40.330238 21287 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:06:40.331395 21287 hybrid_clock.cc:648] HybridClock initialized: now 1786241200331375 us; error 34 us; skew 500 ppm
I20260809 02:06:40.333273 21287 webserver.cc:460] Webserver started at http://127.1.192.195:40745/ using document root <none> and password file <none>
I20260809 02:06:40.333895 21287 fs_manager.cc:359] Metadata directory not provided
I20260809 02:06:40.333961 21287 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:06:40.334169 21287 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:06:40.336248 21287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786240941968412-1795-0/minicluster-data/ts-2/data/instance:
uuid: "daf5c6a1e95a40fa8fd24fd7f1ef449a"
format_stamp: "Formatted at 2026-08-09 02:06:40 on dist-test-slave-ss4n"
server_key: "c1d06b8ca9a3cea2e9dd7002033ece65"
server_key_iv: "34db6501afd1982c681c3431222a110f"
server_key_version: "encryptionkey@0"
I20260809 02:06:40.336838 21287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786240941968412-1795-0/minicluster-data/ts-2/wal/instance:
uuid: "daf5c6a1e95a40fa8fd24fd7f1ef449a"
format_stamp: "Formatted at 2026-08-09 02:06:40 on dist-test-slave-ss4n"
server_key: "c1d06b8ca9a3cea2e9dd7002033ece65"
server_key_iv: "34db6501afd1982c681c3431222a110f"
server_key_version: "encryptionkey@0"
I20260809 02:06:40.340727 21287 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 02:06:40.343325 21302 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:06:40.344660 21287 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 02:06:40.344785 21287 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786240941968412-1795-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786240941968412-1795-0/minicluster-data/ts-2/wal
uuid: "daf5c6a1e95a40fa8fd24fd7f1ef449a"
format_stamp: "Formatted at 2026-08-09 02:06:40 on dist-test-slave-ss4n"
server_key: "c1d06b8ca9a3cea2e9dd7002033ece65"
server_key_iv: "34db6501afd1982c681c3431222a110f"
server_key_version: "encryptionkey@0"
I20260809 02:06:40.344903 21287 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786240941968412-1795-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786240941968412-1795-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786240941968412-1795-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:06:40.359992 21287 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:06:40.360813 21287 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:06:40.361013 21287 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:06:40.361729 21287 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:06:40.363054 21287 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:06:40.363121 21287 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:06:40.363174 21287 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:06:40.363205 21287 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:06:40.376008 21287 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.195:38905
I20260809 02:06:40.376123 21427 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.195:38905 every 8 connection(s)
I20260809 02:06:40.377314 21287 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786240941968412-1795-0/minicluster-data/ts-2/data/info.pb
I20260809 02:06:40.377912 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 21287
I20260809 02:06:40.378034 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786240941968412-1795-0/minicluster-data/ts-2/wal/instance
I20260809 02:06:40.378348 1795 external_mini_cluster.cc:1428] Setting key ebfa41a68389e488c3f75a282914e44f
I20260809 02:06:40.389860 20985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:40.381470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.195:49445 (local address 127.1.192.254:33353)
0809 02:06:40.381679 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0809 02:06:40.381686 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 02:06:40.381705 (+ 19us) server_negotiation.cc:407] Connection header received
0809 02:06:40.383015 (+ 1310us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:06:40.383051 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:06:40.383104 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:06:40.383210 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:06:40.384711 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:40.385506 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:40.387242 (+ 1736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:40.387644 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:40.387776 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:40.389278 (+ 1502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:06:40.389284 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:06:40.389288 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:06:40.389305 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:06:40.389330 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:06:40.389343 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:06:40.389426 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 02:06:40.389606 (+ 180us) server_negotiation.cc:299] Negotiation successful
0809 02:06:40.389681 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260809 02:06:40.391090 21430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:40.380329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:33353 (local address 127.1.192.195:49445)
0809 02:06:40.381196 (+ 867us) negotiation.cc:107] Waiting for socket to connect
0809 02:06:40.381253 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0809 02:06:40.382777 (+ 1524us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:06:40.383399 (+ 622us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:06:40.383406 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:06:40.383484 (+ 78us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:06:40.384052 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:06:40.384075 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:06:40.385634 (+ 1559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:06:40.385641 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:06:40.387107 (+ 1466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:06:40.387118 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:06:40.387871 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:06:40.387878 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:06:40.388115 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:40.388894 (+ 779us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 02:06:40.388897 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:06:40.388977 (+ 80us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 02:06:40.389082 (+ 105us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:06:40.389439 (+ 357us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:06:40.389447 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:06:40.389453 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 02:06:40.389622 (+ 169us) client_negotiation.cc:240] Negotiation successful
0809 02:06:40.389961 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":158,"threads_started":1}
I20260809 02:06:40.393088 21428 heartbeater.cc:347] Connected to a master server at 127.1.192.254:33353
I20260809 02:06:40.393383 21428 heartbeater.cc:464] Registering TS with master...
I20260809 02:06:40.394100 21428 heartbeater.cc:511] Master 127.1.192.254:33353 requested a full tablet report, sending...
I20260809 02:06:40.395172 20943 ts_manager.cc:194] Registered new tserver with Master: daf5c6a1e95a40fa8fd24fd7f1ef449a (127.1.192.195:38905)
I20260809 02:06:40.395952 20943 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.192.195:49445
I20260809 02:06:40.404937 1795 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 02:06:40.411191 20985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:40.406994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44334 (local address 127.1.192.254:33353)
0809 02:06:40.407145 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0809 02:06:40.407149 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:06:40.407354 (+ 205us) server_negotiation.cc:407] Connection header received
0809 02:06:40.407511 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:06:40.407513 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:06:40.407564 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:06:40.407680 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:06:40.407964 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:40.408493 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:40.409426 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:40.409786 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:40.409910 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:40.410750 (+ 840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:06:40.410753 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:06:40.410756 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:06:40.410769 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:06:40.410790 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:06:40.410804 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:06:40.410878 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 02:06:40.410967 (+ 89us) server_negotiation.cc:299] Negotiation successful
0809 02:06:40.411024 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 02:06:40.414544 20943 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44334:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 02:06:40.417691 20943 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:06:40.432592 21439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:40.426112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.195:38905 (local address 127.0.0.1:46264)
0809 02:06:40.426992 (+ 880us) negotiation.cc:107] Waiting for socket to connect
0809 02:06:40.427047 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0809 02:06:40.427278 (+ 231us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:06:40.428040 (+ 762us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:06:40.428046 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:06:40.428084 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:06:40.428240 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:06:40.428248 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:06:40.429712 (+ 1464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:06:40.429715 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:06:40.430975 (+ 1260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:06:40.430985 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:06:40.432043 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:06:40.432047 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:06:40.432160 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:40.432201 (+ 41us) client_negotiation.cc:769] Sending connection context
0809 02:06:40.432300 (+ 99us) client_negotiation.cc:240] Negotiation successful
0809 02:06:40.432412 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":702,"mutex_wait_us":2,"spinlock_wait_cycles":11520,"thread_start_us":108,"threads_started":1}
I20260809 02:06:40.432633 21440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:40.426244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46264 (local address 127.1.192.195:38905)
0809 02:06:40.426749 (+ 505us) server_negotiation.cc:206] Beginning negotiation
0809 02:06:40.426764 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 02:06:40.427155 (+ 391us) server_negotiation.cc:407] Connection header received
0809 02:06:40.427470 (+ 315us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:06:40.427476 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:06:40.427694 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:06:40.427826 (+ 132us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:06:40.428387 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:40.429596 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:40.431095 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:40.431916 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:40.432037 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:40.432277 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0809 02:06:40.432335 (+ 58us) server_negotiation.cc:299] Negotiation successful
0809 02:06:40.432439 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":403,"thread_start_us":166,"threads_started":1}
I20260809 02:06:40.432765 21443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:40.426307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.193:43115 (local address 127.0.0.1:58838)
0809 02:06:40.427156 (+ 849us) negotiation.cc:107] Waiting for socket to connect
0809 02:06:40.427172 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 02:06:40.427298 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:06:40.428042 (+ 744us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:06:40.428046 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:06:40.428078 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:06:40.428201 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:06:40.428206 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:06:40.430099 (+ 1893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:06:40.430103 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:06:40.431174 (+ 1071us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:06:40.431181 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:06:40.432345 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:06:40.432348 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:06:40.432445 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:40.432467 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 02:06:40.432544 (+ 77us) client_negotiation.cc:240] Negotiation successful
0809 02:06:40.432624 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":769,"mutex_wait_us":3,"spinlock_wait_cycles":1792,"thread_start_us":105,"threads_started":1}
I20260809 02:06:40.433336 21442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:40.426509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58838 (local address 127.1.192.193:43115)
0809 02:06:40.426922 (+ 413us) server_negotiation.cc:206] Beginning negotiation
0809 02:06:40.426932 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 02:06:40.427438 (+ 506us) server_negotiation.cc:407] Connection header received
0809 02:06:40.427524 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:06:40.427529 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:06:40.427738 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:06:40.427889 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:06:40.428319 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:40.429585 (+ 1266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:40.431279 (+ 1694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:40.432033 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:40.432127 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:40.432277 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0809 02:06:40.432927 (+ 650us) server_negotiation.cc:299] Negotiation successful
0809 02:06:40.433031 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329,"thread_start_us":118,"threads_started":1}
I20260809 02:06:40.433931 21444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:40.427077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36718 (local address 127.1.192.194:33745)
0809 02:06:40.427513 (+ 436us) server_negotiation.cc:206] Beginning negotiation
0809 02:06:40.427523 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 02:06:40.427763 (+ 240us) server_negotiation.cc:407] Connection header received
0809 02:06:40.427967 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:06:40.427974 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:06:40.428142 (+ 168us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:06:40.428260 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:06:40.428648 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:40.429366 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:40.431874 (+ 2508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:40.432645 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:40.432755 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:40.432936 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0809 02:06:40.433669 (+ 733us) server_negotiation.cc:299] Negotiation successful
0809 02:06:40.433778 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":142,"threads_started":1}
I20260809 02:06:40.434326 21441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:40.426900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.194:33745 (local address 127.0.0.1:36718)
0809 02:06:40.427668 (+ 768us) negotiation.cc:107] Waiting for socket to connect
0809 02:06:40.427681 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:06:40.427827 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:06:40.428439 (+ 612us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:06:40.428441 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:06:40.428455 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:06:40.428520 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:06:40.428524 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:06:40.429528 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:06:40.429531 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:06:40.431758 (+ 2227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:06:40.431763 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:06:40.433217 (+ 1454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:06:40.433220 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:06:40.433305 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:40.433327 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 02:06:40.434118 (+ 791us) client_negotiation.cc:240] Negotiation successful
0809 02:06:40.434208 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":681,"mutex_wait_us":1214,"thread_start_us":97,"threads_started":1}
I20260809 02:06:40.435889 21354 tablet_service.cc:1467] Processing CreateTablet for tablet 35cbd7a0780f40a5929ee5beb44a5b04 (DEFAULT_TABLE table=test-table [id=286479d5af34456aa7fa9e6c734a3069]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:06:40.436614 21212 tablet_service.cc:1467] Processing CreateTablet for tablet 35cbd7a0780f40a5929ee5beb44a5b04 (DEFAULT_TABLE table=test-table [id=286479d5af34456aa7fa9e6c734a3069]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:06:40.436951 21354 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 35cbd7a0780f40a5929ee5beb44a5b04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:06:40.437568 21212 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 35cbd7a0780f40a5929ee5beb44a5b04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:06:40.438226 21072 tablet_service.cc:1467] Processing CreateTablet for tablet 35cbd7a0780f40a5929ee5beb44a5b04 (DEFAULT_TABLE table=test-table [id=286479d5af34456aa7fa9e6c734a3069]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:06:40.439231 21072 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 35cbd7a0780f40a5929ee5beb44a5b04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:06:40.443750 21445 tablet_bootstrap.cc:492] T 35cbd7a0780f40a5929ee5beb44a5b04 P 074ba1ee21c7451fa67494da5b2ee170: Bootstrap starting.
I20260809 02:06:40.445216 21446 tablet_bootstrap.cc:492] T 35cbd7a0780f40a5929ee5beb44a5b04 P daf5c6a1e95a40fa8fd24fd7f1ef449a: Bootstrap starting.
I20260809 02:06:40.446235 21445 tablet_bootstrap.cc:654] T 35cbd7a0780f40a5929ee5beb44a5b04 P 074ba1ee21c7451fa67494da5b2ee170: Neither blocks nor log segments found. Creating new log.
I20260809 02:06:40.447017 21445 log.cc:824] T 35cbd7a0780f40a5929ee5beb44a5b04 P 074ba1ee21c7451fa67494da5b2ee170: Log is configured to *not* fsync() on all Append() calls
I20260809 02:06:40.447273 21447 tablet_bootstrap.cc:492] T 35cbd7a0780f40a5929ee5beb44a5b04 P 8679bc66961e4c98a9e970f36aa7e528: Bootstrap starting.
I20260809 02:06:40.447721 21446 tablet_bootstrap.cc:654] T 35cbd7a0780f40a5929ee5beb44a5b04 P daf5c6a1e95a40fa8fd24fd7f1ef449a: Neither blocks nor log segments found. Creating new log.
I20260809 02:06:40.448674 21446 log.cc:824] T 35cbd7a0780f40a5929ee5beb44a5b04 P daf5c6a1e95a40fa8fd24fd7f1ef449a: Log is configured to *not* fsync() on all Append() calls
I20260809 02:06:40.449497 21445 tablet_bootstrap.cc:492] T 35cbd7a0780f40a5929ee5beb44a5b04 P 074ba1ee21c7451fa67494da5b2ee170: No bootstrap required, opened a new log
I20260809 02:06:40.449684 21447 tablet_bootstrap.cc:654] T 35cbd7a0780f40a5929ee5beb44a5b04 P 8679bc66961e4c98a9e970f36aa7e528: Neither blocks nor log segments found. Creating new log.
I20260809 02:06:40.449734 21445 ts_tablet_manager.cc:1397] T 35cbd7a0780f40a5929ee5beb44a5b04 P 074ba1ee21c7451fa67494da5b2ee170: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260809 02:06:40.450574 21447 log.cc:824] T 35cbd7a0780f40a5929ee5beb44a5b04 P 8679bc66961e4c98a9e970f36aa7e528: Log is configured to *not* fsync() on all Append() calls
I20260809 02:06:40.451608 21446 tablet_bootstrap.cc:492] T 35cbd7a0780f40a5929ee5beb44a5b04 P daf5c6a1e95a40fa8fd24fd7f1ef449a: No bootstrap required, opened a new log
I20260809 02:06:40.451822 21446 ts_tablet_manager.cc:1397] T 35cbd7a0780f40a5929ee5beb44a5b04 P daf5c6a1e95a40fa8fd24fd7f1ef449a: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260809 02:06:40.453387 21447 tablet_bootstrap.cc:492] T 35cbd7a0780f40a5929ee5beb44a5b04 P 8679bc66961e4c98a9e970f36aa7e528: No bootstrap required, opened a new log
I20260809 02:06:40.453624 21447 ts_tablet_manager.cc:1397] T 35cbd7a0780f40a5929ee5beb44a5b04 P 8679bc66961e4c98a9e970f36aa7e528: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.004s
I20260809 02:06:40.454015 21445 raft_consensus.cc:348] T 35cbd7a0780f40a5929ee5beb44a5b04 P 074ba1ee21c7451fa67494da5b2ee170 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "daf5c6a1e95a40fa8fd24fd7f1ef449a" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 38905 } } peers { permanent_uuid: "8679bc66961e4c98a9e970f36aa7e528" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 43115 } } peers { permanent_uuid: "074ba1ee21c7451fa67494da5b2ee170" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 33745 } }
I20260809 02:06:40.454260 21445 raft_consensus.cc:374] T 35cbd7a0780f40a5929ee5beb44a5b04 P 074ba1ee21c7451fa67494da5b2ee170 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:06:40.454319 21445 raft_consensus.cc:729] T 35cbd7a0780f40a5929ee5beb44a5b04 P 074ba1ee21c7451fa67494da5b2ee170 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 074ba1ee21c7451fa67494da5b2ee170, State: Initialized, Role: FOLLOWER
I20260809 02:06:40.454777 21445 consensus_queue.cc:260] T 35cbd7a0780f40a5929ee5beb44a5b04 P 074ba1ee21c7451fa67494da5b2ee170 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "daf5c6a1e95a40fa8fd24fd7f1ef449a" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 38905 } } peers { permanent_uuid: "8679bc66961e4c98a9e970f36aa7e528" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 43115 } } peers { permanent_uuid: "074ba1ee21c7451fa67494da5b2ee170" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 33745 } }
I20260809 02:06:40.455425 21283 heartbeater.cc:503] Master 127.1.192.254:33353 was elected leader, sending a full tablet report...
I20260809 02:06:40.455785 21445 ts_tablet_manager.cc:1428] T 35cbd7a0780f40a5929ee5beb44a5b04 P 074ba1ee21c7451fa67494da5b2ee170: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260809 02:06:40.456732 21446 raft_consensus.cc:348] T 35cbd7a0780f40a5929ee5beb44a5b04 P daf5c6a1e95a40fa8fd24fd7f1ef449a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "daf5c6a1e95a40fa8fd24fd7f1ef449a" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 38905 } } peers { permanent_uuid: "8679bc66961e4c98a9e970f36aa7e528" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 43115 } } peers { permanent_uuid: "074ba1ee21c7451fa67494da5b2ee170" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 33745 } }
I20260809 02:06:40.457001 21446 raft_consensus.cc:374] T 35cbd7a0780f40a5929ee5beb44a5b04 P daf5c6a1e95a40fa8fd24fd7f1ef449a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:06:40.457059 21446 raft_consensus.cc:729] T 35cbd7a0780f40a5929ee5beb44a5b04 P daf5c6a1e95a40fa8fd24fd7f1ef449a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: daf5c6a1e95a40fa8fd24fd7f1ef449a, State: Initialized, Role: FOLLOWER
I20260809 02:06:40.457544 21446 consensus_queue.cc:260] T 35cbd7a0780f40a5929ee5beb44a5b04 P daf5c6a1e95a40fa8fd24fd7f1ef449a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "daf5c6a1e95a40fa8fd24fd7f1ef449a" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 38905 } } peers { permanent_uuid: "8679bc66961e4c98a9e970f36aa7e528" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 43115 } } peers { permanent_uuid: "074ba1ee21c7451fa67494da5b2ee170" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 33745 } }
I20260809 02:06:40.457840 21447 raft_consensus.cc:348] T 35cbd7a0780f40a5929ee5beb44a5b04 P 8679bc66961e4c98a9e970f36aa7e528 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "daf5c6a1e95a40fa8fd24fd7f1ef449a" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 38905 } } peers { permanent_uuid: "8679bc66961e4c98a9e970f36aa7e528" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 43115 } } peers { permanent_uuid: "074ba1ee21c7451fa67494da5b2ee170" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 33745 } }
I20260809 02:06:40.458112 21447 raft_consensus.cc:374] T 35cbd7a0780f40a5929ee5beb44a5b04 P 8679bc66961e4c98a9e970f36aa7e528 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:06:40.458184 21447 raft_consensus.cc:729] T 35cbd7a0780f40a5929ee5beb44a5b04 P 8679bc66961e4c98a9e970f36aa7e528 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8679bc66961e4c98a9e970f36aa7e528, State: Initialized, Role: FOLLOWER
I20260809 02:06:40.458642 21446 ts_tablet_manager.cc:1428] T 35cbd7a0780f40a5929ee5beb44a5b04 P daf5c6a1e95a40fa8fd24fd7f1ef449a: Time spent starting tablet: real 0.007s user 0.004s sys 0.004s
I20260809 02:06:40.458669 21447 consensus_queue.cc:260] T 35cbd7a0780f40a5929ee5beb44a5b04 P 8679bc66961e4c98a9e970f36aa7e528 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "daf5c6a1e95a40fa8fd24fd7f1ef449a" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 38905 } } peers { permanent_uuid: "8679bc66961e4c98a9e970f36aa7e528" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 43115 } } peers { permanent_uuid: "074ba1ee21c7451fa67494da5b2ee170" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 33745 } }
I20260809 02:06:40.459005 21428 heartbeater.cc:503] Master 127.1.192.254:33353 was elected leader, sending a full tablet report...
I20260809 02:06:40.459646 21140 heartbeater.cc:503] Master 127.1.192.254:33353 was elected leader, sending a full tablet report...
I20260809 02:06:40.459637 21447 ts_tablet_manager.cc:1428] T 35cbd7a0780f40a5929ee5beb44a5b04 P 8679bc66961e4c98a9e970f36aa7e528: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260809 02:06:40.482883 21451 raft_consensus.cc:482] T 35cbd7a0780f40a5929ee5beb44a5b04 P 074ba1ee21c7451fa67494da5b2ee170 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:06:40.483124 21451 raft_consensus.cc:504] T 35cbd7a0780f40a5929ee5beb44a5b04 P 074ba1ee21c7451fa67494da5b2ee170 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "daf5c6a1e95a40fa8fd24fd7f1ef449a" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 38905 } } peers { permanent_uuid: "8679bc66961e4c98a9e970f36aa7e528" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 43115 } } peers { permanent_uuid: "074ba1ee21c7451fa67494da5b2ee170" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 33745 } }
I20260809 02:06:40.484393 21451 leader_election.cc:288] T 35cbd7a0780f40a5929ee5beb44a5b04 P 074ba1ee21c7451fa67494da5b2ee170 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers daf5c6a1e95a40fa8fd24fd7f1ef449a (127.1.192.195:38905), 8679bc66961e4c98a9e970f36aa7e528 (127.1.192.193:43115)
I20260809 02:06:40.489665 21285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:40.484706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.193:43115 (local address 127.1.192.194:36355)
0809 02:06:40.484959 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0809 02:06:40.484977 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 02:06:40.485081 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:06:40.485428 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:06:40.485456 (+ 28us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:06:40.485484 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:06:40.485731 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:06:40.485740 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:06:40.486799 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:06:40.486803 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:06:40.488326 (+ 1523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:06:40.488338 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:06:40.489222 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:06:40.489225 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:06:40.489327 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:40.489355 (+ 28us) client_negotiation.cc:769] Sending connection context
0809 02:06:40.489418 (+ 63us) client_negotiation.cc:240] Negotiation successful
0809 02:06:40.489469 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"mutex_wait_us":129,"spinlock_wait_cycles":2432}
I20260809 02:06:40.489687 21442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:40.484829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.194:36355 (local address 127.1.192.193:43115)
0809 02:06:40.485030 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0809 02:06:40.485035 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:06:40.485061 (+ 26us) server_negotiation.cc:407] Connection header received
0809 02:06:40.485207 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:06:40.485209 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:06:40.485286 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:06:40.485375 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:06:40.485876 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:40.486665 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:40.488528 (+ 1863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:40.489025 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:40.489148 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:40.489202 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 02:06:40.489473 (+ 271us) server_negotiation.cc:299] Negotiation successful
0809 02:06:40.489581 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260809 02:06:40.489683 21440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:40.484776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.194:51497 (local address 127.1.192.195:38905)
0809 02:06:40.484928 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0809 02:06:40.484933 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:06:40.485263 (+ 330us) server_negotiation.cc:407] Connection header received
0809 02:06:40.485327 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:06:40.485333 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:06:40.485388 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:06:40.485464 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:06:40.485876 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:40.486667 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:40.488477 (+ 1810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:40.488975 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:40.489100 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:40.489153 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 02:06:40.489431 (+ 278us) server_negotiation.cc:299] Negotiation successful
0809 02:06:40.489544 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 02:06:40.489674 21454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:40.484731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.195:38905 (local address 127.1.192.194:51497)
0809 02:06:40.485155 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0809 02:06:40.485169 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 02:06:40.485262 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:06:40.485520 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:06:40.485522 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:06:40.485546 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:06:40.485738 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:06:40.485744 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:06:40.486799 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:06:40.486803 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:06:40.488326 (+ 1523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:06:40.488338 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:06:40.489091 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:06:40.489095 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:06:40.489221 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:40.489245 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 02:06:40.489361 (+ 116us) client_negotiation.cc:240] Negotiation successful
0809 02:06:40.489449 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":356,"mutex_wait_us":146,"thread_start_us":143,"threads_started":1}
I20260809 02:06:40.490448 21092 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35cbd7a0780f40a5929ee5beb44a5b04" candidate_uuid: "074ba1ee21c7451fa67494da5b2ee170" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8679bc66961e4c98a9e970f36aa7e528" is_pre_election: true
W20260809 02:06:40.490605 21141 tablet.cc:2369] T 35cbd7a0780f40a5929ee5beb44a5b04 P 8679bc66961e4c98a9e970f36aa7e528: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 02:06:40.490810 21092 raft_consensus.cc:2454] T 35cbd7a0780f40a5929ee5beb44a5b04 P 8679bc66961e4c98a9e970f36aa7e528 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 074ba1ee21c7451fa67494da5b2ee170 in term 0.
I20260809 02:06:40.490918 21376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35cbd7a0780f40a5929ee5beb44a5b04" candidate_uuid: "074ba1ee21c7451fa67494da5b2ee170" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "daf5c6a1e95a40fa8fd24fd7f1ef449a" is_pre_election: true
I20260809 02:06:40.491205 21376 raft_consensus.cc:2454] T 35cbd7a0780f40a5929ee5beb44a5b04 P daf5c6a1e95a40fa8fd24fd7f1ef449a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 074ba1ee21c7451fa67494da5b2ee170 in term 0.
I20260809 02:06:40.491339 21161 leader_election.cc:302] T 35cbd7a0780f40a5929ee5beb44a5b04 P 074ba1ee21c7451fa67494da5b2ee170 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 074ba1ee21c7451fa67494da5b2ee170, 8679bc66961e4c98a9e970f36aa7e528; no voters:
I20260809 02:06:40.491631 21451 raft_consensus.cc:2781] T 35cbd7a0780f40a5929ee5beb44a5b04 P 074ba1ee21c7451fa67494da5b2ee170 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:06:40.491746 21451 raft_consensus.cc:482] T 35cbd7a0780f40a5929ee5beb44a5b04 P 074ba1ee21c7451fa67494da5b2ee170 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:06:40.491796 21451 raft_consensus.cc:3037] T 35cbd7a0780f40a5929ee5beb44a5b04 P 074ba1ee21c7451fa67494da5b2ee170 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:06:40.492868 21451 raft_consensus.cc:504] T 35cbd7a0780f40a5929ee5beb44a5b04 P 074ba1ee21c7451fa67494da5b2ee170 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "daf5c6a1e95a40fa8fd24fd7f1ef449a" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 38905 } } peers { permanent_uuid: "8679bc66961e4c98a9e970f36aa7e528" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 43115 } } peers { permanent_uuid: "074ba1ee21c7451fa67494da5b2ee170" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 33745 } }
I20260809 02:06:40.493252 21451 leader_election.cc:288] T 35cbd7a0780f40a5929ee5beb44a5b04 P 074ba1ee21c7451fa67494da5b2ee170 [CANDIDATE]: Term 1 election: Requested vote from peers daf5c6a1e95a40fa8fd24fd7f1ef449a (127.1.192.195:38905), 8679bc66961e4c98a9e970f36aa7e528 (127.1.192.193:43115)
I20260809 02:06:40.493572 21376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35cbd7a0780f40a5929ee5beb44a5b04" candidate_uuid: "074ba1ee21c7451fa67494da5b2ee170" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "daf5c6a1e95a40fa8fd24fd7f1ef449a"
I20260809 02:06:40.493696 21092 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35cbd7a0780f40a5929ee5beb44a5b04" candidate_uuid: "074ba1ee21c7451fa67494da5b2ee170" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8679bc66961e4c98a9e970f36aa7e528"
I20260809 02:06:40.493731 21376 raft_consensus.cc:3037] T 35cbd7a0780f40a5929ee5beb44a5b04 P daf5c6a1e95a40fa8fd24fd7f1ef449a [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:06:40.493829 21092 raft_consensus.cc:3037] T 35cbd7a0780f40a5929ee5beb44a5b04 P 8679bc66961e4c98a9e970f36aa7e528 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:06:40.495119 21376 raft_consensus.cc:2454] T 35cbd7a0780f40a5929ee5beb44a5b04 P daf5c6a1e95a40fa8fd24fd7f1ef449a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 074ba1ee21c7451fa67494da5b2ee170 in term 1.
I20260809 02:06:40.495123 21092 raft_consensus.cc:2454] T 35cbd7a0780f40a5929ee5beb44a5b04 P 8679bc66961e4c98a9e970f36aa7e528 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 074ba1ee21c7451fa67494da5b2ee170 in term 1.
I20260809 02:06:40.495612 21162 leader_election.cc:302] T 35cbd7a0780f40a5929ee5beb44a5b04 P 074ba1ee21c7451fa67494da5b2ee170 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 074ba1ee21c7451fa67494da5b2ee170, daf5c6a1e95a40fa8fd24fd7f1ef449a; no voters:
I20260809 02:06:40.495836 21451 raft_consensus.cc:2781] T 35cbd7a0780f40a5929ee5beb44a5b04 P 074ba1ee21c7451fa67494da5b2ee170 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:06:40.496085 21451 raft_consensus.cc:686] T 35cbd7a0780f40a5929ee5beb44a5b04 P 074ba1ee21c7451fa67494da5b2ee170 [term 1 LEADER]: Becoming Leader. State: Replica: 074ba1ee21c7451fa67494da5b2ee170, State: Running, Role: LEADER
I20260809 02:06:40.496421 21451 consensus_queue.cc:237] T 35cbd7a0780f40a5929ee5beb44a5b04 P 074ba1ee21c7451fa67494da5b2ee170 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "daf5c6a1e95a40fa8fd24fd7f1ef449a" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 38905 } } peers { permanent_uuid: "8679bc66961e4c98a9e970f36aa7e528" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 43115 } } peers { permanent_uuid: "074ba1ee21c7451fa67494da5b2ee170" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 33745 } }
I20260809 02:06:40.499601 20945 catalog_manager.cc:5495] T 35cbd7a0780f40a5929ee5beb44a5b04 P 074ba1ee21c7451fa67494da5b2ee170 reported cstate change: term changed from 0 to 1, leader changed from <none> to 074ba1ee21c7451fa67494da5b2ee170 (127.1.192.194). New cstate: current_term: 1 leader_uuid: "074ba1ee21c7451fa67494da5b2ee170" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "daf5c6a1e95a40fa8fd24fd7f1ef449a" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 38905 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8679bc66961e4c98a9e970f36aa7e528" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 43115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "074ba1ee21c7451fa67494da5b2ee170" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 33745 } health_report { overall_health: HEALTHY } } }
I20260809 02:06:40.513000 21444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:40.509593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36726 (local address 127.1.192.194:33745)
0809 02:06:40.509752 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0809 02:06:40.509762 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 02:06:40.509773 (+ 11us) server_negotiation.cc:407] Connection header received
0809 02:06:40.509841 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:06:40.509843 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:06:40.509895 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:06:40.510004 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:06:40.510252 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:40.510743 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:40.511565 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:40.511783 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:40.511891 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:40.512189 (+ 298us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:06:40.512600 (+ 411us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:06:40.512732 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0809 02:06:40.512795 (+ 63us) server_negotiation.cc:299] Negotiation successful
0809 02:06:40.512851 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 02:06:40.518800 21092 raft_consensus.cc:1261] T 35cbd7a0780f40a5929ee5beb44a5b04 P 8679bc66961e4c98a9e970f36aa7e528 [term 1 FOLLOWER]: Refusing update from remote peer 074ba1ee21c7451fa67494da5b2ee170: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:06:40.518800 21376 raft_consensus.cc:1261] T 35cbd7a0780f40a5929ee5beb44a5b04 P daf5c6a1e95a40fa8fd24fd7f1ef449a [term 1 FOLLOWER]: Refusing update from remote peer 074ba1ee21c7451fa67494da5b2ee170: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:06:40.519479 21451 consensus_queue.cc:1035] T 35cbd7a0780f40a5929ee5beb44a5b04 P 074ba1ee21c7451fa67494da5b2ee170 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8679bc66961e4c98a9e970f36aa7e528" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 43115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:06:40.519677 21455 consensus_queue.cc:1035] T 35cbd7a0780f40a5929ee5beb44a5b04 P 074ba1ee21c7451fa67494da5b2ee170 [LEADER]: Connected to new peer: Peer: permanent_uuid: "daf5c6a1e95a40fa8fd24fd7f1ef449a" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 38905 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:06:40.536693 20942 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44334:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:06:40.536872 20942 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44334:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:06:40.539316 20942 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 08e8346ff27c476884ad09902069a35a: Sending DeleteTablet for 3 replicas of tablet 35cbd7a0780f40a5929ee5beb44a5b04
I20260809 02:06:40.540088 21354 tablet_service.cc:1514] Processing DeleteTablet for tablet 35cbd7a0780f40a5929ee5beb44a5b04 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:06:40 UTC) from {username='slave'} at 127.0.0.1:46264
I20260809 02:06:40.540189 21212 tablet_service.cc:1514] Processing DeleteTablet for tablet 35cbd7a0780f40a5929ee5beb44a5b04 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:06:40 UTC) from {username='slave'} at 127.0.0.1:36718
I20260809 02:06:40.540221 21072 tablet_service.cc:1514] Processing DeleteTablet for tablet 35cbd7a0780f40a5929ee5beb44a5b04 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:06:40 UTC) from {username='slave'} at 127.0.0.1:58838
I20260809 02:06:40.540663 21469 tablet_replica.cc:330] T 35cbd7a0780f40a5929ee5beb44a5b04 P 8679bc66961e4c98a9e970f36aa7e528: stopping tablet replica
I20260809 02:06:40.540685 21468 tablet_replica.cc:330] T 35cbd7a0780f40a5929ee5beb44a5b04 P daf5c6a1e95a40fa8fd24fd7f1ef449a: stopping tablet replica
I20260809 02:06:40.540987 21469 raft_consensus.cc:2229] T 35cbd7a0780f40a5929ee5beb44a5b04 P 8679bc66961e4c98a9e970f36aa7e528 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:06:40.540992 21468 raft_consensus.cc:2229] T 35cbd7a0780f40a5929ee5beb44a5b04 P daf5c6a1e95a40fa8fd24fd7f1ef449a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:06:40.541308 21469 raft_consensus.cc:2258] T 35cbd7a0780f40a5929ee5beb44a5b04 P 8679bc66961e4c98a9e970f36aa7e528 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:06:40.541308 21468 raft_consensus.cc:2258] T 35cbd7a0780f40a5929ee5beb44a5b04 P daf5c6a1e95a40fa8fd24fd7f1ef449a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:06:40.541662 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 21006
I20260809 02:06:40.541715 21470 tablet_replica.cc:330] T 35cbd7a0780f40a5929ee5beb44a5b04 P 074ba1ee21c7451fa67494da5b2ee170: stopping tablet replica
I20260809 02:06:40.541944 21470 raft_consensus.cc:2229] T 35cbd7a0780f40a5929ee5beb44a5b04 P 074ba1ee21c7451fa67494da5b2ee170 [term 1 LEADER]: Raft consensus shutting down.
I20260809 02:06:40.543128 21470 raft_consensus.cc:2258] T 35cbd7a0780f40a5929ee5beb44a5b04 P 074ba1ee21c7451fa67494da5b2ee170 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:06:40.544651 21468 ts_tablet_manager.cc:1905] T 35cbd7a0780f40a5929ee5beb44a5b04 P daf5c6a1e95a40fa8fd24fd7f1ef449a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:06:40.545217 21470 ts_tablet_manager.cc:1905] T 35cbd7a0780f40a5929ee5beb44a5b04 P 074ba1ee21c7451fa67494da5b2ee170: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:06:40.548318 21470 ts_tablet_manager.cc:1918] T 35cbd7a0780f40a5929ee5beb44a5b04 P 074ba1ee21c7451fa67494da5b2ee170: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:06:40.548418 21470 log.cc:1196] T 35cbd7a0780f40a5929ee5beb44a5b04 P 074ba1ee21c7451fa67494da5b2ee170: Deleting WAL directory at /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786240941968412-1795-0/minicluster-data/ts-1/wal/wals/35cbd7a0780f40a5929ee5beb44a5b04
I20260809 02:06:40.548489 21468 ts_tablet_manager.cc:1918] T 35cbd7a0780f40a5929ee5beb44a5b04 P daf5c6a1e95a40fa8fd24fd7f1ef449a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:06:40.548586 21468 log.cc:1196] T 35cbd7a0780f40a5929ee5beb44a5b04 P daf5c6a1e95a40fa8fd24fd7f1ef449a: Deleting WAL directory at /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786240941968412-1795-0/minicluster-data/ts-2/wal/wals/35cbd7a0780f40a5929ee5beb44a5b04
I20260809 02:06:40.548801 21470 ts_tablet_manager.cc:1939] T 35cbd7a0780f40a5929ee5beb44a5b04 P 074ba1ee21c7451fa67494da5b2ee170: Deleting consensus metadata
I20260809 02:06:40.548902 21468 ts_tablet_manager.cc:1939] T 35cbd7a0780f40a5929ee5beb44a5b04 P daf5c6a1e95a40fa8fd24fd7f1ef449a: Deleting consensus metadata
I20260809 02:06:40.550220 20928 catalog_manager.cc:4841] TS daf5c6a1e95a40fa8fd24fd7f1ef449a (127.1.192.195:38905): tablet 35cbd7a0780f40a5929ee5beb44a5b04 (table test-table [id=286479d5af34456aa7fa9e6c734a3069]) successfully deleted
I20260809 02:06:40.550263 20927 catalog_manager.cc:4841] TS 074ba1ee21c7451fa67494da5b2ee170 (127.1.192.194:33745): tablet 35cbd7a0780f40a5929ee5beb44a5b04 (table test-table [id=286479d5af34456aa7fa9e6c734a3069]) successfully deleted
W20260809 02:06:40.553223 20927 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.192.193:43115 (error 108)
I20260809 02:06:40.554237 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 21143
I20260809 02:06:40.554440 21441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:40.554110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.193:43115 (local address 127.0.0.1:58844)
0809 02:06:40.554241 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0809 02:06:40.554330 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.193:43115: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260809 02:06:40.556115 20927 catalog_manager.cc:4584] TS 8679bc66961e4c98a9e970f36aa7e528 (127.1.192.193:43115): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 35cbd7a0780f40a5929ee5beb44a5b04: Network error: Client connection negotiation failed: client connection to 127.1.192.193:43115: connect: Connection refused (error 111)
I20260809 02:06:40.561839 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 21287
I20260809 02:06:40.568648 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 20909
2026-08-09T02:06:40Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1045 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786240941968412-1795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:06:40 dist-test-slave-ss4n krb5kdc[21474](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:06:40 dist-test-slave-ss4n krb5kdc[21474](info): set up 2 sockets
Aug 09 02:06:40 dist-test-slave-ss4n krb5kdc[21474](info): commencing operation
krb5kdc: starting...
W20260809 02:06:41.659282 1795 mini_kdc.cc:121] Time spent starting KDC: real 1.068s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786240941968412-1795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786240941968412-1795-0/krb5kdc/test-user.keytab.
Aug 09 02:06:41 dist-test-slave-ss4n krb5kdc[21474](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241201, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:06:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:06:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.192.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.192.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.192.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab.
Entry for principal kudu/127.1.192.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab.
I20260809 02:06:41.805934 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786240941968412-1795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786240941968412-1795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786240941968412-1795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.192.254:46131
--webserver_interface=127.1.192.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab
--principal=kudu/127.1.192.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:42273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.192.254:46131
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:06:41.920235 21489 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:06:41.920562 21489 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:06:41.920644 21489 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:06:41.925894 21489 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:06:41.925978 21489 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:06:41.926015 21489 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:06:41.926052 21489 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:06:41.926086 21489 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:06:41.931633 21489 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:42273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786240941968412-1795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786240941968412-1795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.192.254:46131
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab
--principal=kudu/127.1.192.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.192.254:46131
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.192.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21489
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786240941968412-1795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:06:41.932832 21489 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:06:41.933923 21489 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:06:41.943008 21498 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:06:41.943065 21489 server_base.cc:1034] running on GCE node
W20260809 02:06:41.943032 21494 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:06:41.943003 21495 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:06:41.943879 21489 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:06:41.945101 21489 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:06:41.946274 21489 hybrid_clock.cc:648] HybridClock initialized: now 1786241201946240 us; error 46 us; skew 500 ppm
Aug 09 02:06:41 dist-test-slave-ss4n krb5kdc[21474](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241201, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:06:41.954746 21489 init.cc:377] Logged in from keytab as kudu/127.1.192.254@KRBTEST.COM (short username kudu)
I20260809 02:06:41.955993 21489 webserver.cc:460] Webserver started at http://127.1.192.254:38071/ using document root <none> and password file <none>
I20260809 02:06:41.956660 21489 fs_manager.cc:359] Metadata directory not provided
I20260809 02:06:41.956738 21489 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:06:41.957000 21489 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:06:41.959484 21489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786240941968412-1795-0/minicluster-data/master-0/data/instance:
uuid: "0fed2967a2c74bb3a37a56a15d404035"
format_stamp: "Formatted at 2026-08-09 02:06:41 on dist-test-slave-ss4n"
server_key: "259f64d23befccb8ca22c242c662076f"
server_key_iv: "3db07b68f1e0032e6b80828fe4ecbdd5"
server_key_version: "encryptionkey@0"
I20260809 02:06:41.960251 21489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786240941968412-1795-0/minicluster-data/master-0/wal/instance:
uuid: "0fed2967a2c74bb3a37a56a15d404035"
format_stamp: "Formatted at 2026-08-09 02:06:41 on dist-test-slave-ss4n"
server_key: "259f64d23befccb8ca22c242c662076f"
server_key_iv: "3db07b68f1e0032e6b80828fe4ecbdd5"
server_key_version: "encryptionkey@0"
I20260809 02:06:41.964867 21489 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.005s
I20260809 02:06:41.967846 21506 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:06:41.969588 21489 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 02:06:41.969750 21489 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786240941968412-1795-0/minicluster-data/master-0/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786240941968412-1795-0/minicluster-data/master-0/wal
uuid: "0fed2967a2c74bb3a37a56a15d404035"
format_stamp: "Formatted at 2026-08-09 02:06:41 on dist-test-slave-ss4n"
server_key: "259f64d23befccb8ca22c242c662076f"
server_key_iv: "3db07b68f1e0032e6b80828fe4ecbdd5"
server_key_version: "encryptionkey@0"
I20260809 02:06:41.969933 21489 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786240941968412-1795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786240941968412-1795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786240941968412-1795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:06:41.994187 21489 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:06:41.997232 21489 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:06:41.997449 21489 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:06:42.005043 21489 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.254:46131
I20260809 02:06:42.005065 21561 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.254:46131 every 8 connection(s)
I20260809 02:06:42.006096 21489 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
I20260809 02:06:42.009143 21562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:06:42.013968 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 21489
I20260809 02:06:42.014150 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786240941968412-1795-0/minicluster-data/master-0/wal/instance
I20260809 02:06:42.014430 1795 external_mini_cluster.cc:1428] Setting key 0fb54ef811c5e692e008e868ec482d45
I20260809 02:06:42.015652 21562 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0fed2967a2c74bb3a37a56a15d404035: Bootstrap starting.
I20260809 02:06:42.018246 21562 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0fed2967a2c74bb3a37a56a15d404035: Neither blocks nor log segments found. Creating new log.
I20260809 02:06:42.019069 21562 log.cc:824] T 00000000000000000000000000000000 P 0fed2967a2c74bb3a37a56a15d404035: Log is configured to *not* fsync() on all Append() calls
I20260809 02:06:42.021126 21562 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0fed2967a2c74bb3a37a56a15d404035: No bootstrap required, opened a new log
Aug 09 02:06:42 dist-test-slave-ss4n krb5kdc[21474](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241201, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.192.254@KRBTEST.COM
I20260809 02:06:42.025434 21562 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0fed2967a2c74bb3a37a56a15d404035 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fed2967a2c74bb3a37a56a15d404035" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 46131 } }
I20260809 02:06:42.025740 21562 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0fed2967a2c74bb3a37a56a15d404035 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:06:42.025827 21562 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0fed2967a2c74bb3a37a56a15d404035 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fed2967a2c74bb3a37a56a15d404035, State: Initialized, Role: FOLLOWER
I20260809 02:06:42.026280 21562 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0fed2967a2c74bb3a37a56a15d404035 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fed2967a2c74bb3a37a56a15d404035" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 46131 } }
I20260809 02:06:42.026409 21562 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0fed2967a2c74bb3a37a56a15d404035 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:06:42.026512 21562 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0fed2967a2c74bb3a37a56a15d404035 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:06:42.026633 21562 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0fed2967a2c74bb3a37a56a15d404035 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:06:42.027628 21562 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0fed2967a2c74bb3a37a56a15d404035 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fed2967a2c74bb3a37a56a15d404035" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 46131 } }
I20260809 02:06:42.028070 21562 leader_election.cc:302] T 00000000000000000000000000000000 P 0fed2967a2c74bb3a37a56a15d404035 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0fed2967a2c74bb3a37a56a15d404035; no voters:
I20260809 02:06:42.028455 21562 leader_election.cc:288] T 00000000000000000000000000000000 P 0fed2967a2c74bb3a37a56a15d404035 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:06:42.028641 21567 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0fed2967a2c74bb3a37a56a15d404035 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:06:42.028898 21567 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0fed2967a2c74bb3a37a56a15d404035 [term 1 LEADER]: Becoming Leader. State: Replica: 0fed2967a2c74bb3a37a56a15d404035, State: Running, Role: LEADER
I20260809 02:06:42.029213 21567 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0fed2967a2c74bb3a37a56a15d404035 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fed2967a2c74bb3a37a56a15d404035" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 46131 } }
I20260809 02:06:42.029991 21562 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0fed2967a2c74bb3a37a56a15d404035 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:06:42.030915 21568 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0fed2967a2c74bb3a37a56a15d404035 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0fed2967a2c74bb3a37a56a15d404035" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fed2967a2c74bb3a37a56a15d404035" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 46131 } } }
I20260809 02:06:42.031034 21568 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0fed2967a2c74bb3a37a56a15d404035 [sys.catalog]: This master's current role is: LEADER
I20260809 02:06:42.031227 21565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:42.015959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47636 (local address 127.1.192.254:46131)
0809 02:06:42.016502 (+ 543us) server_negotiation.cc:206] Beginning negotiation
0809 02:06:42.016518 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 02:06:42.016555 (+ 37us) server_negotiation.cc:407] Connection header received
0809 02:06:42.017238 (+ 683us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:06:42.017256 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:06:42.017730 (+ 474us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:06:42.018071 (+ 341us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:06:42.018929 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:42.019986 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:42.020882 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:42.021415 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:42.021617 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:42.024458 (+ 2841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:06:42.024463 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:06:42.024476 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:06:42.024509 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:06:42.026848 (+ 2339us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:06:42.027321 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:06:42.027327 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:06:42.027334 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:06:42.027461 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:06:42.027797 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:06:42.027812 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:06:42.027814 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:06:42.028506 (+ 692us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:06:42.028718 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0809 02:06:42.029684 (+ 966us) server_negotiation.cc:299] Negotiation successful
0809 02:06:42.030039 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":364,"thread_start_us":154,"threads_started":1}
I20260809 02:06:42.031625 21569 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0fed2967a2c74bb3a37a56a15d404035 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0fed2967a2c74bb3a37a56a15d404035. Latest consensus state: current_term: 1 leader_uuid: "0fed2967a2c74bb3a37a56a15d404035" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fed2967a2c74bb3a37a56a15d404035" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 46131 } } }
I20260809 02:06:42.031776 21569 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0fed2967a2c74bb3a37a56a15d404035 [sys.catalog]: This master's current role is: LEADER
W20260809 02:06:42.035202 21584 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0fed2967a2c74bb3a37a56a15d404035: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 02:06:42.035291 21584 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 02:06:42.035365 21576 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:06:42.036324 21576 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:06:42.042281 21576 catalog_manager.cc:1347] Generated new cluster ID: b0dfd9910a074ae2b9a3b629e65881af
I20260809 02:06:42.042383 21576 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:06:42.048632 21576 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:06:42.049746 21576 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:06:42.056408 21576 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0fed2967a2c74bb3a37a56a15d404035: Generated new TSK 0
I20260809 02:06:42.057255 21576 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.192.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.192.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.192.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.193.keytab.
Entry for principal kudu/127.1.192.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.193.keytab.
I20260809 02:06:42.093382 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786240941968412-1795-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786240941968412-1795-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786240941968412-1795-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786240941968412-1795-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.192.193:0
--local_ip_for_outbound_sockets=127.1.192.193
--webserver_interface=127.1.192.193
--webserver_port=0
--tserver_master_addrs=127.1.192.254:46131
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.193.keytab
--principal=kudu/127.1.192.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:42273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:06:42.225174 21589 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:06:42.225476 21589 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:06:42.225561 21589 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:06:42.229709 21589 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:06:42.229844 21589 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:06:42.229929 21589 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.192.193
I20260809 02:06:42.235334 21589 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:42273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786240941968412-1795-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786240941968412-1795-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.193.keytab
--principal=kudu/127.1.192.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.192.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786240941968412-1795-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.192.193
--webserver_port=0
--tserver_master_addrs=127.1.192.254:46131
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21589
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.192.193
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786240941968412-1795-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:06:42.236377 21589 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:06:42.237376 21589 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:06:42.247035 21598 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:06:42.247251 21589 server_base.cc:1034] running on GCE node
W20260809 02:06:42.247045 21596 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:06:42.247042 21595 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:06:42.247821 21589 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:06:42.248461 21589 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:06:42.249657 21589 hybrid_clock.cc:648] HybridClock initialized: now 1786241202249616 us; error 53 us; skew 500 ppm
Aug 09 02:06:42 dist-test-slave-ss4n krb5kdc[21474](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241202, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:06:42.252578 21589 init.cc:377] Logged in from keytab as kudu/127.1.192.193@KRBTEST.COM (short username kudu)
I20260809 02:06:42.253433 21589 webserver.cc:460] Webserver started at http://127.1.192.193:41919/ using document root <none> and password file <none>
I20260809 02:06:42.254019 21589 fs_manager.cc:359] Metadata directory not provided
I20260809 02:06:42.254125 21589 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:06:42.254359 21589 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:06:42.256448 21589 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786240941968412-1795-0/minicluster-data/ts-0/data/instance:
uuid: "3cf2b36e4e13486da203ba04d65a3aef"
format_stamp: "Formatted at 2026-08-09 02:06:42 on dist-test-slave-ss4n"
server_key: "e66a66da5e30b1b45954a1e0e6926305"
server_key_iv: "15b4aae71d4963edd0de18f8140757db"
server_key_version: "encryptionkey@0"
I20260809 02:06:42.257073 21589 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786240941968412-1795-0/minicluster-data/ts-0/wal/instance:
uuid: "3cf2b36e4e13486da203ba04d65a3aef"
format_stamp: "Formatted at 2026-08-09 02:06:42 on dist-test-slave-ss4n"
server_key: "e66a66da5e30b1b45954a1e0e6926305"
server_key_iv: "15b4aae71d4963edd0de18f8140757db"
server_key_version: "encryptionkey@0"
I20260809 02:06:42.261184 21589 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260809 02:06:42.263950 21605 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:06:42.265187 21589 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 02:06:42.265321 21589 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786240941968412-1795-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786240941968412-1795-0/minicluster-data/ts-0/wal
uuid: "3cf2b36e4e13486da203ba04d65a3aef"
format_stamp: "Formatted at 2026-08-09 02:06:42 on dist-test-slave-ss4n"
server_key: "e66a66da5e30b1b45954a1e0e6926305"
server_key_iv: "15b4aae71d4963edd0de18f8140757db"
server_key_version: "encryptionkey@0"
I20260809 02:06:42.265440 21589 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786240941968412-1795-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786240941968412-1795-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786240941968412-1795-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:06:42.296262 21589 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:06:42.299919 21589 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:06:42.300150 21589 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:06:42.300800 21589 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:06:42.301743 21589 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:06:42.301812 21589 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:06:42.301888 21589 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:06:42.301935 21589 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:06:42.311514 21589 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.193:44153
I20260809 02:06:42.311537 21725 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.193:44153 every 8 connection(s)
I20260809 02:06:42.312532 21589 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786240941968412-1795-0/minicluster-data/ts-0/data/info.pb
Aug 09 02:06:42 dist-test-slave-ss4n krb5kdc[21474](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241202, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.193@KRBTEST.COM for kudu/127.1.192.254@KRBTEST.COM
I20260809 02:06:42.322855 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 21589
I20260809 02:06:42.323010 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786240941968412-1795-0/minicluster-data/ts-0/wal/instance
I20260809 02:06:42.323359 1795 external_mini_cluster.cc:1428] Setting key cc404cf0741a9b9e737e8bcaccb8492f
I20260809 02:06:42.326750 21565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:42.314367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.193:34699 (local address 127.1.192.254:46131)
0809 02:06:42.314509 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0809 02:06:42.314513 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:06:42.315386 (+ 873us) server_negotiation.cc:407] Connection header received
0809 02:06:42.316289 (+ 903us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:06:42.316293 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:06:42.316352 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:06:42.316456 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:06:42.317682 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:42.318225 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:42.319375 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:42.319611 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:42.319730 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:42.322429 (+ 2699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:06:42.322432 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:06:42.322434 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:06:42.322459 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:06:42.324369 (+ 1910us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:06:42.325018 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:06:42.325022 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:06:42.325024 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:06:42.325113 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:06:42.325549 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:06:42.325552 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:06:42.325553 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:06:42.325740 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:06:42.325821 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 02:06:42.326515 (+ 694us) server_negotiation.cc:299] Negotiation successful
0809 02:06:42.326626 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 02:06:42.327888 21729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:42.314766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:46131 (local address 127.1.192.193:34699)
0809 02:06:42.315230 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0809 02:06:42.315273 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 02:06:42.316074 (+ 801us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:06:42.316623 (+ 549us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:06:42.316630 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:06:42.316985 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:06:42.317466 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:06:42.317536 (+ 70us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:06:42.318362 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:06:42.318365 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:06:42.319249 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:06:42.319258 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:06:42.319775 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:06:42.319780 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:06:42.319953 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:42.320573 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:06:42.320589 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:06:42.322252 (+ 1663us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:06:42.324482 (+ 2230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:06:42.324487 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:06:42.324500 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:06:42.324892 (+ 392us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:06:42.325216 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:06:42.325220 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:06:42.325222 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:06:42.325421 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:06:42.325858 (+ 437us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:06:42.325867 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:06:42.326242 (+ 375us) client_negotiation.cc:769] Sending connection context
0809 02:06:42.326461 (+ 219us) client_negotiation.cc:240] Negotiation successful
0809 02:06:42.326855 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":116,"threads_started":1}
I20260809 02:06:42.329227 21726 heartbeater.cc:347] Connected to a master server at 127.1.192.254:46131
I20260809 02:06:42.329556 21726 heartbeater.cc:464] Registering TS with master...
I20260809 02:06:42.330295 21726 heartbeater.cc:511] Master 127.1.192.254:46131 requested a full tablet report, sending...
I20260809 02:06:42.332351 21524 ts_manager.cc:194] Registered new tserver with Master: 3cf2b36e4e13486da203ba04d65a3aef (127.1.192.193:44153)
I20260809 02:06:42.334237 21524 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.192.193@KRBTEST.COM'} at 127.1.192.193:34699
WARNING: no policy specified for kudu/127.1.192.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.192.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.192.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.194.keytab.
Entry for principal kudu/127.1.192.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.194.keytab.
I20260809 02:06:42.357426 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786240941968412-1795-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786240941968412-1795-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786240941968412-1795-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786240941968412-1795-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.192.194:0
--local_ip_for_outbound_sockets=127.1.192.194
--webserver_interface=127.1.192.194
--webserver_port=0
--tserver_master_addrs=127.1.192.254:46131
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.194.keytab
--principal=kudu/127.1.192.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:42273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:06:42.477111 21732 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:06:42.477440 21732 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:06:42.477550 21732 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:06:42.482031 21732 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:06:42.482136 21732 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:06:42.482245 21732 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.192.194
I20260809 02:06:42.488627 21732 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:42273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786240941968412-1795-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786240941968412-1795-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.194.keytab
--principal=kudu/127.1.192.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.192.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786240941968412-1795-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.192.194
--webserver_port=0
--tserver_master_addrs=127.1.192.254:46131
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21732
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.192.194
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786240941968412-1795-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:06:42.489930 21732 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:06:42.491091 21732 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:06:42.501904 21740 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:06:42.501931 21737 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:06:42.501931 21738 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:06:42.502120 21732 server_base.cc:1034] running on GCE node
I20260809 02:06:42.502676 21732 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:06:42.503325 21732 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:06:42.504525 21732 hybrid_clock.cc:648] HybridClock initialized: now 1786241202504500 us; error 37 us; skew 500 ppm
Aug 09 02:06:42 dist-test-slave-ss4n krb5kdc[21474](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241202, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:06:42.507359 21732 init.cc:377] Logged in from keytab as kudu/127.1.192.194@KRBTEST.COM (short username kudu)
I20260809 02:06:42.508220 21732 webserver.cc:460] Webserver started at http://127.1.192.194:43041/ using document root <none> and password file <none>
I20260809 02:06:42.508791 21732 fs_manager.cc:359] Metadata directory not provided
I20260809 02:06:42.508901 21732 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:06:42.509140 21732 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:06:42.511263 21732 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786240941968412-1795-0/minicluster-data/ts-1/data/instance:
uuid: "7dce7e884fe2443781c4b797f720064e"
format_stamp: "Formatted at 2026-08-09 02:06:42 on dist-test-slave-ss4n"
server_key: "3cac017d058a6b17a18ce44b984aaac4"
server_key_iv: "081096c6bf234a4d75c45b7930fe2334"
server_key_version: "encryptionkey@0"
I20260809 02:06:42.511842 21732 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786240941968412-1795-0/minicluster-data/ts-1/wal/instance:
uuid: "7dce7e884fe2443781c4b797f720064e"
format_stamp: "Formatted at 2026-08-09 02:06:42 on dist-test-slave-ss4n"
server_key: "3cac017d058a6b17a18ce44b984aaac4"
server_key_iv: "081096c6bf234a4d75c45b7930fe2334"
server_key_version: "encryptionkey@0"
I20260809 02:06:42.515692 21732 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 02:06:42.518230 21747 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:06:42.519572 21732 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 02:06:42.519709 21732 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786240941968412-1795-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786240941968412-1795-0/minicluster-data/ts-1/wal
uuid: "7dce7e884fe2443781c4b797f720064e"
format_stamp: "Formatted at 2026-08-09 02:06:42 on dist-test-slave-ss4n"
server_key: "3cac017d058a6b17a18ce44b984aaac4"
server_key_iv: "081096c6bf234a4d75c45b7930fe2334"
server_key_version: "encryptionkey@0"
I20260809 02:06:42.519827 21732 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786240941968412-1795-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786240941968412-1795-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786240941968412-1795-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:06:42.547170 21732 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:06:42.550420 21732 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:06:42.550629 21732 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:06:42.551249 21732 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:06:42.552088 21732 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:06:42.552166 21732 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:06:42.552237 21732 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:06:42.552286 21732 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:06:42.561555 21732 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.194:44689
I20260809 02:06:42.561605 21867 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.194:44689 every 8 connection(s)
I20260809 02:06:42.562496 21732 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786240941968412-1795-0/minicluster-data/ts-1/data/info.pb
I20260809 02:06:42.565611 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 21732
I20260809 02:06:42.565752 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786240941968412-1795-0/minicluster-data/ts-1/wal/instance
I20260809 02:06:42.566067 1795 external_mini_cluster.cc:1428] Setting key 16862b572fa0413d8ba6ce61b26080ee
Aug 09 02:06:42 dist-test-slave-ss4n krb5kdc[21474](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241202, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.194@KRBTEST.COM for kudu/127.1.192.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.192.195@KRBTEST.COM; defaulting to no policy
I20260809 02:06:42.578287 21565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:42.564576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.194:40771 (local address 127.1.192.254:46131)
0809 02:06:42.564712 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0809 02:06:42.564715 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:06:42.565731 (+ 1016us) server_negotiation.cc:407] Connection header received
0809 02:06:42.566904 (+ 1173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:06:42.566908 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:06:42.566969 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:06:42.567094 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:06:42.568581 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:42.569202 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:42.570625 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:42.570864 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:42.570975 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:42.573883 (+ 2908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:06:42.573890 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:06:42.573896 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:06:42.573933 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:06:42.575867 (+ 1934us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:06:42.576433 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:06:42.576440 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:06:42.576443 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:06:42.576539 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:06:42.576897 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:06:42.576901 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:06:42.576903 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:06:42.577172 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:06:42.577289 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0809 02:06:42.577904 (+ 615us) server_negotiation.cc:299] Negotiation successful
0809 02:06:42.578064 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 02:06:42.579262 21872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:42.564991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:46131 (local address 127.1.192.194:40771)
0809 02:06:42.565544 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0809 02:06:42.565608 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0809 02:06:42.566652 (+ 1044us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:06:42.567237 (+ 585us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:06:42.567244 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:06:42.567709 (+ 465us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:06:42.568382 (+ 673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:06:42.568404 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:06:42.569335 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:06:42.569341 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:06:42.570499 (+ 1158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:06:42.570508 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:06:42.571014 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:06:42.571018 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:06:42.571192 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:42.571830 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:06:42.571855 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:06:42.573704 (+ 1849us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:06:42.575996 (+ 2292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:06:42.576002 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:06:42.576015 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:06:42.576308 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:06:42.576640 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:06:42.576642 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:06:42.576644 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:06:42.576790 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:06:42.577286 (+ 496us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:06:42.577292 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:06:42.577630 (+ 338us) client_negotiation.cc:769] Sending connection context
0809 02:06:42.577799 (+ 169us) client_negotiation.cc:240] Negotiation successful
0809 02:06:42.578179 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":127,"threads_started":1}
I20260809 02:06:42.580664 21868 heartbeater.cc:347] Connected to a master server at 127.1.192.254:46131
I20260809 02:06:42.580945 21868 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.192.195@KRBTEST.COM" created.
I20260809 02:06:42.581635 21868 heartbeater.cc:511] Master 127.1.192.254:46131 requested a full tablet report, sending...
I20260809 02:06:42.582890 21524 ts_manager.cc:194] Registered new tserver with Master: 7dce7e884fe2443781c4b797f720064e (127.1.192.194:44689)
I20260809 02:06:42.583523 21524 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.192.194@KRBTEST.COM'} at 127.1.192.194:40771
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.192.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.195.keytab.
Entry for principal kudu/127.1.192.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.195.keytab.
I20260809 02:06:42.600523 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786240941968412-1795-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786240941968412-1795-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786240941968412-1795-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786240941968412-1795-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.192.195:0
--local_ip_for_outbound_sockets=127.1.192.195
--webserver_interface=127.1.192.195
--webserver_port=0
--tserver_master_addrs=127.1.192.254:46131
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.195.keytab
--principal=kudu/127.1.192.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:42273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:06:42.718696 21875 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:06:42.719015 21875 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:06:42.719064 21875 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:06:42.723560 21875 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:06:42.723654 21875 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:06:42.723774 21875 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.192.195
I20260809 02:06:42.730316 21875 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:42273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786240941968412-1795-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786240941968412-1795-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.195.keytab
--principal=kudu/127.1.192.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.192.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786240941968412-1795-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.192.195
--webserver_port=0
--tserver_master_addrs=127.1.192.254:46131
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21875
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.192.195
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786240941968412-1795-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:06:42.731710 21875 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:06:42.732849 21875 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:06:42.743382 21884 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:06:42.743472 21875 server_base.cc:1034] running on GCE node
W20260809 02:06:42.743407 21882 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:06:42.743392 21881 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:06:42.744119 21875 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:06:42.744823 21875 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:06:42.746002 21875 hybrid_clock.cc:648] HybridClock initialized: now 1786241202745979 us; error 36 us; skew 500 ppm
Aug 09 02:06:42 dist-test-slave-ss4n krb5kdc[21474](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241202, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:06:42.749064 21875 init.cc:377] Logged in from keytab as kudu/127.1.192.195@KRBTEST.COM (short username kudu)
I20260809 02:06:42.750097 21875 webserver.cc:460] Webserver started at http://127.1.192.195:42499/ using document root <none> and password file <none>
I20260809 02:06:42.750689 21875 fs_manager.cc:359] Metadata directory not provided
I20260809 02:06:42.750782 21875 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:06:42.751012 21875 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:06:42.753153 21875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786240941968412-1795-0/minicluster-data/ts-2/data/instance:
uuid: "47a423cc1a8f4ad08ed3117ca42a3de3"
format_stamp: "Formatted at 2026-08-09 02:06:42 on dist-test-slave-ss4n"
server_key: "fb30604e5887ad3e71edf4f46b48f45d"
server_key_iv: "00363c62673535dcadc43fbd4dc31a5a"
server_key_version: "encryptionkey@0"
I20260809 02:06:42.753765 21875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786240941968412-1795-0/minicluster-data/ts-2/wal/instance:
uuid: "47a423cc1a8f4ad08ed3117ca42a3de3"
format_stamp: "Formatted at 2026-08-09 02:06:42 on dist-test-slave-ss4n"
server_key: "fb30604e5887ad3e71edf4f46b48f45d"
server_key_iv: "00363c62673535dcadc43fbd4dc31a5a"
server_key_version: "encryptionkey@0"
I20260809 02:06:42.758080 21875 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260809 02:06:42.761171 21891 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:06:42.762444 21875 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 02:06:42.762583 21875 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786240941968412-1795-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786240941968412-1795-0/minicluster-data/ts-2/wal
uuid: "47a423cc1a8f4ad08ed3117ca42a3de3"
format_stamp: "Formatted at 2026-08-09 02:06:42 on dist-test-slave-ss4n"
server_key: "fb30604e5887ad3e71edf4f46b48f45d"
server_key_iv: "00363c62673535dcadc43fbd4dc31a5a"
server_key_version: "encryptionkey@0"
I20260809 02:06:42.762712 21875 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786240941968412-1795-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786240941968412-1795-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786240941968412-1795-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:06:42.778039 21875 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:06:42.781240 21875 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:06:42.781450 21875 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:06:42.782167 21875 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:06:42.782984 21875 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:06:42.783031 21875 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:06:42.783070 21875 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:06:42.783116 21875 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:06:42.791937 21875 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.195:39383
I20260809 02:06:42.791966 22010 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.195:39383 every 8 connection(s)
I20260809 02:06:42.792829 21875 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786240941968412-1795-0/minicluster-data/ts-2/data/info.pb
I20260809 02:06:42.798225 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 21875
I20260809 02:06:42.798363 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786240941968412-1795-0/minicluster-data/ts-2/wal/instance
I20260809 02:06:42.798671 1795 external_mini_cluster.cc:1428] Setting key d11a4a6472ad87145bc7dede4162de77
Aug 09 02:06:42 dist-test-slave-ss4n krb5kdc[21474](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241202, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.195@KRBTEST.COM for kudu/127.1.192.254@KRBTEST.COM
I20260809 02:06:42.806838 21565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:42.794731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.195:33721 (local address 127.1.192.254:46131)
0809 02:06:42.794978 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0809 02:06:42.794982 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:06:42.795632 (+ 650us) server_negotiation.cc:407] Connection header received
0809 02:06:42.796543 (+ 911us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:06:42.796547 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:06:42.796606 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:06:42.796716 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:06:42.797946 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:42.798745 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:42.800395 (+ 1650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:42.800603 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:42.800732 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:42.803283 (+ 2551us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:06:42.803286 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:06:42.803288 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:06:42.803313 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:06:42.804665 (+ 1352us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:06:42.805239 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:06:42.805242 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:06:42.805243 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:06:42.805321 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:06:42.805739 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:06:42.805742 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:06:42.805743 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:06:42.805953 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:06:42.806058 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0809 02:06:42.806602 (+ 544us) server_negotiation.cc:299] Negotiation successful
0809 02:06:42.806716 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155}
I20260809 02:06:42.807603 22013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:42.795055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:46131 (local address 127.1.192.195:33721)
0809 02:06:42.795485 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0809 02:06:42.795527 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 02:06:42.796341 (+ 814us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:06:42.796885 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:06:42.796892 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:06:42.797272 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:06:42.797806 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:06:42.797821 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:06:42.798897 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:06:42.798907 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:06:42.800283 (+ 1376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:06:42.800295 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:06:42.800788 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:06:42.800794 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:06:42.800978 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:42.801545 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:06:42.801565 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:06:42.803124 (+ 1559us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:06:42.804779 (+ 1655us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:06:42.804784 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:06:42.804796 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:06:42.805143 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:06:42.805402 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:06:42.805404 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:06:42.805405 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:06:42.805634 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:06:42.806070 (+ 436us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:06:42.806075 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:06:42.806305 (+ 230us) client_negotiation.cc:769] Sending connection context
0809 02:06:42.806458 (+ 153us) client_negotiation.cc:240] Negotiation successful
0809 02:06:42.806748 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":104,"threads_started":1}
I20260809 02:06:42.808816 22011 heartbeater.cc:347] Connected to a master server at 127.1.192.254:46131
I20260809 02:06:42.809070 22011 heartbeater.cc:464] Registering TS with master...
I20260809 02:06:42.809592 22011 heartbeater.cc:511] Master 127.1.192.254:46131 requested a full tablet report, sending...
I20260809 02:06:42.810673 21524 ts_manager.cc:194] Registered new tserver with Master: 47a423cc1a8f4ad08ed3117ca42a3de3 (127.1.192.195:39383)
I20260809 02:06:42.811216 21524 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.192.195@KRBTEST.COM'} at 127.1.192.195:33721
I20260809 02:06:42.812778 1795 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 02:06:42.822765 21565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:42.815063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47638 (local address 127.1.192.254:46131)
0809 02:06:42.815218 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0809 02:06:42.815221 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:06:42.815401 (+ 180us) server_negotiation.cc:407] Connection header received
0809 02:06:42.815501 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:06:42.815503 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:06:42.815541 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:06:42.815621 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:06:42.816336 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:42.816854 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:42.817838 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:42.818062 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:42.818137 (+ 75us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:42.819672 (+ 1535us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:06:42.819675 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:06:42.819677 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:06:42.819699 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:06:42.821127 (+ 1428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:06:42.821671 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:06:42.821674 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:06:42.821675 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:06:42.821750 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:06:42.822023 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:06:42.822026 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:06:42.822027 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:06:42.822253 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:06:42.822367 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0809 02:06:42.822571 (+ 204us) server_negotiation.cc:299] Negotiation successful
0809 02:06:42.822646 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260809 02:06:42.825948 21524 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47638:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 02:06:42.828918 21524 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:06:42.844115 22023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:42.837952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49318 (local address 127.1.192.193:44153)
0809 02:06:42.839403 (+ 1451us) server_negotiation.cc:206] Beginning negotiation
0809 02:06:42.839409 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:06:42.839427 (+ 18us) server_negotiation.cc:407] Connection header received
0809 02:06:42.839488 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:06:42.839492 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:06:42.839635 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:06:42.839816 (+ 181us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:06:42.840151 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:42.841321 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:42.842870 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:42.843420 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:42.843554 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:42.843712 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0809 02:06:42.843848 (+ 136us) server_negotiation.cc:299] Negotiation successful
0809 02:06:42.843945 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1354,"thread_start_us":105,"threads_started":1}
I20260809 02:06:42.844485 22022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:42.837806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.193:44153 (local address 127.0.0.1:49318)
0809 02:06:42.838459 (+ 653us) negotiation.cc:107] Waiting for socket to connect
0809 02:06:42.838687 (+ 228us) client_negotiation.cc:174] Beginning negotiation
0809 02:06:42.838819 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:06:42.839786 (+ 967us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:06:42.839789 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:06:42.839823 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:06:42.840022 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:06:42.840032 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:06:42.841446 (+ 1414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:06:42.841450 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:06:42.842738 (+ 1288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:06:42.842748 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:06:42.843553 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:06:42.843557 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:06:42.843664 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:42.843701 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 02:06:42.843818 (+ 117us) client_negotiation.cc:240] Negotiation successful
0809 02:06:42.843952 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":485,"mutex_wait_us":95,"spinlock_wait_cycles":8576,"thread_start_us":81,"threads_started":1}
I20260809 02:06:42.844537 22024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:42.837981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.195:39383 (local address 127.0.0.1:56320)
0809 02:06:42.838773 (+ 792us) negotiation.cc:107] Waiting for socket to connect
0809 02:06:42.838790 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 02:06:42.838883 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:06:42.839781 (+ 898us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:06:42.839787 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:06:42.839824 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:06:42.840207 (+ 383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:06:42.840212 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:06:42.841188 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:06:42.841193 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:06:42.842666 (+ 1473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:06:42.842679 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:06:42.843950 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:06:42.843954 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:06:42.844148 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:42.844217 (+ 69us) client_negotiation.cc:769] Sending connection context
0809 02:06:42.844286 (+ 69us) client_negotiation.cc:240] Negotiation successful
0809 02:06:42.844394 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":717,"mutex_wait_us":408,"spinlock_wait_cycles":6528,"thread_start_us":100,"threads_started":1}
I20260809 02:06:42.844904 22027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:42.838266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.194:44689 (local address 127.0.0.1:33814)
0809 02:06:42.839050 (+ 784us) negotiation.cc:107] Waiting for socket to connect
0809 02:06:42.839062 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:06:42.839161 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:06:42.839896 (+ 735us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:06:42.839898 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:06:42.839914 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:06:42.840022 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:06:42.840029 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:06:42.841466 (+ 1437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:06:42.841469 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:06:42.842712 (+ 1243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:06:42.842722 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:06:42.843950 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:06:42.843954 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:06:42.844143 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:42.844215 (+ 72us) client_negotiation.cc:769] Sending connection context
0809 02:06:42.844287 (+ 72us) client_negotiation.cc:240] Negotiation successful
0809 02:06:42.844788 (+ 501us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":721,"mutex_wait_us":521,"spinlock_wait_cycles":1280,"thread_start_us":105,"threads_started":1}
I20260809 02:06:42.844959 22028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:42.838605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56320 (local address 127.1.192.195:39383)
0809 02:06:42.839072 (+ 467us) server_negotiation.cc:206] Beginning negotiation
0809 02:06:42.839082 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 02:06:42.839101 (+ 19us) server_negotiation.cc:407] Connection header received
0809 02:06:42.839186 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:06:42.839193 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:06:42.839405 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:06:42.839564 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:06:42.840313 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:42.841059 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:42.842803 (+ 1744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:42.843643 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:42.843753 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:42.843884 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0809 02:06:42.844721 (+ 837us) server_negotiation.cc:299] Negotiation successful
0809 02:06:42.844820 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":385,"thread_start_us":156,"threads_started":1}
I20260809 02:06:42.845220 22029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:42.838728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33814 (local address 127.1.192.194:44689)
0809 02:06:42.839342 (+ 614us) server_negotiation.cc:206] Beginning negotiation
0809 02:06:42.839354 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 02:06:42.839375 (+ 21us) server_negotiation.cc:407] Connection header received
0809 02:06:42.839450 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:06:42.839456 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:06:42.839754 (+ 298us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:06:42.839883 (+ 129us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:06:42.840141 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:42.841319 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:42.842864 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:42.843652 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:42.843753 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:42.843884 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0809 02:06:42.844425 (+ 541us) server_negotiation.cc:299] Negotiation successful
0809 02:06:42.844510 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":515,"thread_start_us":136,"threads_started":1}
I20260809 02:06:42.847379 21656 tablet_service.cc:1467] Processing CreateTablet for tablet 5af77087969c43aca136b6609222eecd (DEFAULT_TABLE table=test-table [id=9fc4a6f4b2b44b529a8f534a45c08f62]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:06:42.847391 21941 tablet_service.cc:1467] Processing CreateTablet for tablet 5af77087969c43aca136b6609222eecd (DEFAULT_TABLE table=test-table [id=9fc4a6f4b2b44b529a8f534a45c08f62]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:06:42.848299 21656 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5af77087969c43aca136b6609222eecd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:06:42.848308 21941 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5af77087969c43aca136b6609222eecd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:06:42.855268 22030 tablet_bootstrap.cc:492] T 5af77087969c43aca136b6609222eecd P 3cf2b36e4e13486da203ba04d65a3aef: Bootstrap starting.
I20260809 02:06:42.855340 22031 tablet_bootstrap.cc:492] T 5af77087969c43aca136b6609222eecd P 47a423cc1a8f4ad08ed3117ca42a3de3: Bootstrap starting.
I20260809 02:06:42.856118 21799 tablet_service.cc:1467] Processing CreateTablet for tablet 5af77087969c43aca136b6609222eecd (DEFAULT_TABLE table=test-table [id=9fc4a6f4b2b44b529a8f534a45c08f62]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:06:42.857075 21799 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5af77087969c43aca136b6609222eecd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:06:42.857681 22030 tablet_bootstrap.cc:654] T 5af77087969c43aca136b6609222eecd P 3cf2b36e4e13486da203ba04d65a3aef: Neither blocks nor log segments found. Creating new log.
I20260809 02:06:42.858140 22031 tablet_bootstrap.cc:654] T 5af77087969c43aca136b6609222eecd P 47a423cc1a8f4ad08ed3117ca42a3de3: Neither blocks nor log segments found. Creating new log.
I20260809 02:06:42.858646 22030 log.cc:824] T 5af77087969c43aca136b6609222eecd P 3cf2b36e4e13486da203ba04d65a3aef: Log is configured to *not* fsync() on all Append() calls
I20260809 02:06:42.858930 22031 log.cc:824] T 5af77087969c43aca136b6609222eecd P 47a423cc1a8f4ad08ed3117ca42a3de3: Log is configured to *not* fsync() on all Append() calls
I20260809 02:06:42.860888 22030 tablet_bootstrap.cc:492] T 5af77087969c43aca136b6609222eecd P 3cf2b36e4e13486da203ba04d65a3aef: No bootstrap required, opened a new log
I20260809 02:06:42.860909 22031 tablet_bootstrap.cc:492] T 5af77087969c43aca136b6609222eecd P 47a423cc1a8f4ad08ed3117ca42a3de3: No bootstrap required, opened a new log
I20260809 02:06:42.861104 22030 ts_tablet_manager.cc:1397] T 5af77087969c43aca136b6609222eecd P 3cf2b36e4e13486da203ba04d65a3aef: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260809 02:06:42.861112 22031 ts_tablet_manager.cc:1397] T 5af77087969c43aca136b6609222eecd P 47a423cc1a8f4ad08ed3117ca42a3de3: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260809 02:06:42.864481 22035 tablet_bootstrap.cc:492] T 5af77087969c43aca136b6609222eecd P 7dce7e884fe2443781c4b797f720064e: Bootstrap starting.
I20260809 02:06:42.864686 22030 raft_consensus.cc:348] T 5af77087969c43aca136b6609222eecd P 3cf2b36e4e13486da203ba04d65a3aef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3cf2b36e4e13486da203ba04d65a3aef" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 44153 } } peers { permanent_uuid: "47a423cc1a8f4ad08ed3117ca42a3de3" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 39383 } } peers { permanent_uuid: "7dce7e884fe2443781c4b797f720064e" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 44689 } }
I20260809 02:06:42.864957 22030 raft_consensus.cc:374] T 5af77087969c43aca136b6609222eecd P 3cf2b36e4e13486da203ba04d65a3aef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:06:42.865013 22030 raft_consensus.cc:729] T 5af77087969c43aca136b6609222eecd P 3cf2b36e4e13486da203ba04d65a3aef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3cf2b36e4e13486da203ba04d65a3aef, State: Initialized, Role: FOLLOWER
I20260809 02:06:42.865020 22031 raft_consensus.cc:348] T 5af77087969c43aca136b6609222eecd P 47a423cc1a8f4ad08ed3117ca42a3de3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3cf2b36e4e13486da203ba04d65a3aef" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 44153 } } peers { permanent_uuid: "47a423cc1a8f4ad08ed3117ca42a3de3" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 39383 } } peers { permanent_uuid: "7dce7e884fe2443781c4b797f720064e" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 44689 } }
I20260809 02:06:42.865294 22031 raft_consensus.cc:374] T 5af77087969c43aca136b6609222eecd P 47a423cc1a8f4ad08ed3117ca42a3de3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:06:42.865357 22031 raft_consensus.cc:729] T 5af77087969c43aca136b6609222eecd P 47a423cc1a8f4ad08ed3117ca42a3de3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47a423cc1a8f4ad08ed3117ca42a3de3, State: Initialized, Role: FOLLOWER
I20260809 02:06:42.865530 22030 consensus_queue.cc:260] T 5af77087969c43aca136b6609222eecd P 3cf2b36e4e13486da203ba04d65a3aef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3cf2b36e4e13486da203ba04d65a3aef" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 44153 } } peers { permanent_uuid: "47a423cc1a8f4ad08ed3117ca42a3de3" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 39383 } } peers { permanent_uuid: "7dce7e884fe2443781c4b797f720064e" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 44689 } }
I20260809 02:06:42.865829 22031 consensus_queue.cc:260] T 5af77087969c43aca136b6609222eecd P 47a423cc1a8f4ad08ed3117ca42a3de3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3cf2b36e4e13486da203ba04d65a3aef" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 44153 } } peers { permanent_uuid: "47a423cc1a8f4ad08ed3117ca42a3de3" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 39383 } } peers { permanent_uuid: "7dce7e884fe2443781c4b797f720064e" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 44689 } }
I20260809 02:06:42.866334 21726 heartbeater.cc:503] Master 127.1.192.254:46131 was elected leader, sending a full tablet report...
I20260809 02:06:42.866604 22030 ts_tablet_manager.cc:1428] T 5af77087969c43aca136b6609222eecd P 3cf2b36e4e13486da203ba04d65a3aef: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260809 02:06:42.866865 22031 ts_tablet_manager.cc:1428] T 5af77087969c43aca136b6609222eecd P 47a423cc1a8f4ad08ed3117ca42a3de3: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260809 02:06:42.867185 22011 heartbeater.cc:503] Master 127.1.192.254:46131 was elected leader, sending a full tablet report...
I20260809 02:06:42.869884 22035 tablet_bootstrap.cc:654] T 5af77087969c43aca136b6609222eecd P 7dce7e884fe2443781c4b797f720064e: Neither blocks nor log segments found. Creating new log.
I20260809 02:06:42.870947 22035 log.cc:824] T 5af77087969c43aca136b6609222eecd P 7dce7e884fe2443781c4b797f720064e: Log is configured to *not* fsync() on all Append() calls
I20260809 02:06:42.873178 22035 tablet_bootstrap.cc:492] T 5af77087969c43aca136b6609222eecd P 7dce7e884fe2443781c4b797f720064e: No bootstrap required, opened a new log
I20260809 02:06:42.873363 22035 ts_tablet_manager.cc:1397] T 5af77087969c43aca136b6609222eecd P 7dce7e884fe2443781c4b797f720064e: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.000s
I20260809 02:06:42.876708 22035 raft_consensus.cc:348] T 5af77087969c43aca136b6609222eecd P 7dce7e884fe2443781c4b797f720064e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3cf2b36e4e13486da203ba04d65a3aef" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 44153 } } peers { permanent_uuid: "47a423cc1a8f4ad08ed3117ca42a3de3" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 39383 } } peers { permanent_uuid: "7dce7e884fe2443781c4b797f720064e" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 44689 } }
I20260809 02:06:42.877038 22035 raft_consensus.cc:374] T 5af77087969c43aca136b6609222eecd P 7dce7e884fe2443781c4b797f720064e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:06:42.877097 22035 raft_consensus.cc:729] T 5af77087969c43aca136b6609222eecd P 7dce7e884fe2443781c4b797f720064e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dce7e884fe2443781c4b797f720064e, State: Initialized, Role: FOLLOWER
I20260809 02:06:42.877558 22035 consensus_queue.cc:260] T 5af77087969c43aca136b6609222eecd P 7dce7e884fe2443781c4b797f720064e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3cf2b36e4e13486da203ba04d65a3aef" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 44153 } } peers { permanent_uuid: "47a423cc1a8f4ad08ed3117ca42a3de3" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 39383 } } peers { permanent_uuid: "7dce7e884fe2443781c4b797f720064e" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 44689 } }
I20260809 02:06:42.878285 21868 heartbeater.cc:503] Master 127.1.192.254:46131 was elected leader, sending a full tablet report...
I20260809 02:06:42.878506 22035 ts_tablet_manager.cc:1428] T 5af77087969c43aca136b6609222eecd P 7dce7e884fe2443781c4b797f720064e: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260809 02:06:43.035209 22037 raft_consensus.cc:482] T 5af77087969c43aca136b6609222eecd P 47a423cc1a8f4ad08ed3117ca42a3de3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:06:43.035449 22037 raft_consensus.cc:504] T 5af77087969c43aca136b6609222eecd P 47a423cc1a8f4ad08ed3117ca42a3de3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3cf2b36e4e13486da203ba04d65a3aef" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 44153 } } peers { permanent_uuid: "47a423cc1a8f4ad08ed3117ca42a3de3" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 39383 } } peers { permanent_uuid: "7dce7e884fe2443781c4b797f720064e" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 44689 } }
I20260809 02:06:43.036696 22037 leader_election.cc:288] T 5af77087969c43aca136b6609222eecd P 47a423cc1a8f4ad08ed3117ca42a3de3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3cf2b36e4e13486da203ba04d65a3aef (127.1.192.193:44153), 7dce7e884fe2443781c4b797f720064e (127.1.192.194:44689)
I20260809 02:06:43.042069 22029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:43.037270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.195:34709 (local address 127.1.192.194:44689)
0809 02:06:43.037426 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0809 02:06:43.037433 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 02:06:43.037454 (+ 21us) server_negotiation.cc:407] Connection header received
0809 02:06:43.037618 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:06:43.037621 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:06:43.037673 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:06:43.037741 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:06:43.038118 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:43.038911 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:43.040697 (+ 1786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:43.041327 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:43.041458 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:43.041535 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 02:06:43.041858 (+ 323us) server_negotiation.cc:299] Negotiation successful
0809 02:06:43.041930 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 02:06:43.042078 22023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:43.037251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.195:58341 (local address 127.1.192.193:44153)
0809 02:06:43.037393 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0809 02:06:43.037397 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:06:43.037838 (+ 441us) server_negotiation.cc:407] Connection header received
0809 02:06:43.037973 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:06:43.037975 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:06:43.038015 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:06:43.038103 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:06:43.038364 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:43.039095 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:43.040728 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:43.041341 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:43.041458 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:43.041535 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 02:06:43.041858 (+ 323us) server_negotiation.cc:299] Negotiation successful
0809 02:06:43.041930 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 02:06:43.042121 22013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:43.037124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.194:44689 (local address 127.1.192.195:34709)
0809 02:06:43.037284 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0809 02:06:43.037308 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 02:06:43.037431 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:06:43.037793 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:06:43.037799 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:06:43.037826 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:06:43.037990 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:06:43.038003 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:06:43.039029 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:06:43.039032 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:06:43.040538 (+ 1506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:06:43.040550 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:06:43.041458 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:06:43.041462 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:06:43.041666 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:43.041736 (+ 70us) client_negotiation.cc:769] Sending connection context
0809 02:06:43.041831 (+ 95us) client_negotiation.cc:240] Negotiation successful
0809 02:06:43.041935 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44,"mutex_wait_us":137,"spinlock_wait_cycles":1280}
I20260809 02:06:43.042361 22040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:43.037124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.193:44153 (local address 127.1.192.195:58341)
0809 02:06:43.037750 (+ 626us) negotiation.cc:107] Waiting for socket to connect
0809 02:06:43.037766 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 02:06:43.037870 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:06:43.038130 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:06:43.038133 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:06:43.038156 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:06:43.038247 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:06:43.038251 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:06:43.039187 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:06:43.039190 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:06:43.040537 (+ 1347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:06:43.040616 (+ 79us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:06:43.041458 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:06:43.041462 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:06:43.041737 (+ 275us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:43.041757 (+ 20us) client_negotiation.cc:769] Sending connection context
0809 02:06:43.041832 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 02:06:43.041935 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":540,"mutex_wait_us":226,"thread_start_us":148,"threads_started":1}
I20260809 02:06:43.042872 21819 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5af77087969c43aca136b6609222eecd" candidate_uuid: "47a423cc1a8f4ad08ed3117ca42a3de3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dce7e884fe2443781c4b797f720064e" is_pre_election: true
I20260809 02:06:43.043067 21677 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5af77087969c43aca136b6609222eecd" candidate_uuid: "47a423cc1a8f4ad08ed3117ca42a3de3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3cf2b36e4e13486da203ba04d65a3aef" is_pre_election: true
I20260809 02:06:43.043200 21819 raft_consensus.cc:2454] T 5af77087969c43aca136b6609222eecd P 7dce7e884fe2443781c4b797f720064e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 47a423cc1a8f4ad08ed3117ca42a3de3 in term 0.
I20260809 02:06:43.043394 21677 raft_consensus.cc:2454] T 5af77087969c43aca136b6609222eecd P 3cf2b36e4e13486da203ba04d65a3aef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 47a423cc1a8f4ad08ed3117ca42a3de3 in term 0.
I20260809 02:06:43.043823 21894 leader_election.cc:302] T 5af77087969c43aca136b6609222eecd P 47a423cc1a8f4ad08ed3117ca42a3de3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 47a423cc1a8f4ad08ed3117ca42a3de3, 7dce7e884fe2443781c4b797f720064e; no voters:
W20260809 02:06:43.043833 22012 tablet.cc:2369] T 5af77087969c43aca136b6609222eecd P 47a423cc1a8f4ad08ed3117ca42a3de3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 02:06:43.044152 22037 raft_consensus.cc:2781] T 5af77087969c43aca136b6609222eecd P 47a423cc1a8f4ad08ed3117ca42a3de3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:06:43.044241 22037 raft_consensus.cc:482] T 5af77087969c43aca136b6609222eecd P 47a423cc1a8f4ad08ed3117ca42a3de3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:06:43.044297 22037 raft_consensus.cc:3037] T 5af77087969c43aca136b6609222eecd P 47a423cc1a8f4ad08ed3117ca42a3de3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:06:43.045368 22037 raft_consensus.cc:504] T 5af77087969c43aca136b6609222eecd P 47a423cc1a8f4ad08ed3117ca42a3de3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3cf2b36e4e13486da203ba04d65a3aef" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 44153 } } peers { permanent_uuid: "47a423cc1a8f4ad08ed3117ca42a3de3" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 39383 } } peers { permanent_uuid: "7dce7e884fe2443781c4b797f720064e" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 44689 } }
I20260809 02:06:43.045799 22037 leader_election.cc:288] T 5af77087969c43aca136b6609222eecd P 47a423cc1a8f4ad08ed3117ca42a3de3 [CANDIDATE]: Term 1 election: Requested vote from peers 3cf2b36e4e13486da203ba04d65a3aef (127.1.192.193:44153), 7dce7e884fe2443781c4b797f720064e (127.1.192.194:44689)
I20260809 02:06:43.046197 21819 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5af77087969c43aca136b6609222eecd" candidate_uuid: "47a423cc1a8f4ad08ed3117ca42a3de3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dce7e884fe2443781c4b797f720064e"
I20260809 02:06:43.046206 21677 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5af77087969c43aca136b6609222eecd" candidate_uuid: "47a423cc1a8f4ad08ed3117ca42a3de3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3cf2b36e4e13486da203ba04d65a3aef"
I20260809 02:06:43.046335 21819 raft_consensus.cc:3037] T 5af77087969c43aca136b6609222eecd P 7dce7e884fe2443781c4b797f720064e [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:06:43.046355 21677 raft_consensus.cc:3037] T 5af77087969c43aca136b6609222eecd P 3cf2b36e4e13486da203ba04d65a3aef [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:06:43.047402 21819 raft_consensus.cc:2454] T 5af77087969c43aca136b6609222eecd P 7dce7e884fe2443781c4b797f720064e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 47a423cc1a8f4ad08ed3117ca42a3de3 in term 1.
I20260809 02:06:43.047603 21677 raft_consensus.cc:2454] T 5af77087969c43aca136b6609222eecd P 3cf2b36e4e13486da203ba04d65a3aef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 47a423cc1a8f4ad08ed3117ca42a3de3 in term 1.
I20260809 02:06:43.047739 21894 leader_election.cc:302] T 5af77087969c43aca136b6609222eecd P 47a423cc1a8f4ad08ed3117ca42a3de3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 47a423cc1a8f4ad08ed3117ca42a3de3, 7dce7e884fe2443781c4b797f720064e; no voters:
I20260809 02:06:43.047930 22037 raft_consensus.cc:2781] T 5af77087969c43aca136b6609222eecd P 47a423cc1a8f4ad08ed3117ca42a3de3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:06:43.048177 22037 raft_consensus.cc:686] T 5af77087969c43aca136b6609222eecd P 47a423cc1a8f4ad08ed3117ca42a3de3 [term 1 LEADER]: Becoming Leader. State: Replica: 47a423cc1a8f4ad08ed3117ca42a3de3, State: Running, Role: LEADER
I20260809 02:06:43.048521 22037 consensus_queue.cc:237] T 5af77087969c43aca136b6609222eecd P 47a423cc1a8f4ad08ed3117ca42a3de3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3cf2b36e4e13486da203ba04d65a3aef" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 44153 } } peers { permanent_uuid: "47a423cc1a8f4ad08ed3117ca42a3de3" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 39383 } } peers { permanent_uuid: "7dce7e884fe2443781c4b797f720064e" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 44689 } }
I20260809 02:06:43.051687 21524 catalog_manager.cc:5495] T 5af77087969c43aca136b6609222eecd P 47a423cc1a8f4ad08ed3117ca42a3de3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 47a423cc1a8f4ad08ed3117ca42a3de3 (127.1.192.195). New cstate: current_term: 1 leader_uuid: "47a423cc1a8f4ad08ed3117ca42a3de3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3cf2b36e4e13486da203ba04d65a3aef" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 44153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "47a423cc1a8f4ad08ed3117ca42a3de3" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 39383 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7dce7e884fe2443781c4b797f720064e" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 44689 } health_report { overall_health: UNKNOWN } } }
W20260809 02:06:43.063730 21869 tablet.cc:2369] T 5af77087969c43aca136b6609222eecd P 7dce7e884fe2443781c4b797f720064e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 02:06:43.063970 21727 tablet.cc:2369] T 5af77087969c43aca136b6609222eecd P 3cf2b36e4e13486da203ba04d65a3aef: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 02:06:43.084347 22028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:43.080071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56324 (local address 127.1.192.195:39383)
0809 02:06:43.080229 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0809 02:06:43.080234 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:06:43.080252 (+ 18us) server_negotiation.cc:407] Connection header received
0809 02:06:43.080394 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:06:43.080398 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:06:43.080491 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:06:43.080637 (+ 146us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:06:43.081059 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:43.081667 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:43.082629 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:43.082877 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:43.082997 (+ 120us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:43.083352 (+ 355us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:06:43.083830 (+ 478us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:06:43.084009 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0809 02:06:43.084135 (+ 126us) server_negotiation.cc:299] Negotiation successful
0809 02:06:43.084182 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 02:06:43.090564 21677 raft_consensus.cc:1261] T 5af77087969c43aca136b6609222eecd P 3cf2b36e4e13486da203ba04d65a3aef [term 1 FOLLOWER]: Refusing update from remote peer 47a423cc1a8f4ad08ed3117ca42a3de3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:06:43.090564 21819 raft_consensus.cc:1261] T 5af77087969c43aca136b6609222eecd P 7dce7e884fe2443781c4b797f720064e [term 1 FOLLOWER]: Refusing update from remote peer 47a423cc1a8f4ad08ed3117ca42a3de3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:06:43.091377 22041 consensus_queue.cc:1035] T 5af77087969c43aca136b6609222eecd P 47a423cc1a8f4ad08ed3117ca42a3de3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7dce7e884fe2443781c4b797f720064e" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 44689 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:06:43.091596 22037 consensus_queue.cc:1035] T 5af77087969c43aca136b6609222eecd P 47a423cc1a8f4ad08ed3117ca42a3de3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3cf2b36e4e13486da203ba04d65a3aef" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 44153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:06:43.099216 22046 mvcc.cc:205] Tried to move back new op lower bound from 7316443967851261952 to 7316443967689322496. Current Snapshot: MvccSnapshot[applied={T|T < 7316443967851261952}]
I20260809 02:06:43.100232 22044 mvcc.cc:205] Tried to move back new op lower bound from 7316443967851261952 to 7316443967689322496. Current Snapshot: MvccSnapshot[applied={T|T < 7316443967851261952}]
I20260809 02:06:43.109746 21524 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47638:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:06:43.109927 21524 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47638:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:06:43.112643 21524 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0fed2967a2c74bb3a37a56a15d404035: Sending DeleteTablet for 3 replicas of tablet 5af77087969c43aca136b6609222eecd
I20260809 02:06:43.113649 21656 tablet_service.cc:1514] Processing DeleteTablet for tablet 5af77087969c43aca136b6609222eecd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:06:43 UTC) from {username='kudu', principal='kudu/127.1.192.254@KRBTEST.COM'} at 127.0.0.1:49318
I20260809 02:06:43.113932 21799 tablet_service.cc:1514] Processing DeleteTablet for tablet 5af77087969c43aca136b6609222eecd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:06:43 UTC) from {username='kudu', principal='kudu/127.1.192.254@KRBTEST.COM'} at 127.0.0.1:33814
I20260809 02:06:43.114277 21941 tablet_service.cc:1514] Processing DeleteTablet for tablet 5af77087969c43aca136b6609222eecd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:06:43 UTC) from {username='kudu', principal='kudu/127.1.192.254@KRBTEST.COM'} at 127.0.0.1:56320
I20260809 02:06:43.114507 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 21589
I20260809 02:06:43.114534 22055 tablet_replica.cc:330] T 5af77087969c43aca136b6609222eecd P 7dce7e884fe2443781c4b797f720064e: stopping tablet replica
I20260809 02:06:43.116422 22055 raft_consensus.cc:2229] T 5af77087969c43aca136b6609222eecd P 7dce7e884fe2443781c4b797f720064e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:06:43.116851 22055 raft_consensus.cc:2258] T 5af77087969c43aca136b6609222eecd P 7dce7e884fe2443781c4b797f720064e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:06:43.118990 22055 ts_tablet_manager.cc:1905] T 5af77087969c43aca136b6609222eecd P 7dce7e884fe2443781c4b797f720064e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:06:43.119827 22056 tablet_replica.cc:330] T 5af77087969c43aca136b6609222eecd P 47a423cc1a8f4ad08ed3117ca42a3de3: stopping tablet replica
I20260809 02:06:43.120201 22056 raft_consensus.cc:2229] T 5af77087969c43aca136b6609222eecd P 47a423cc1a8f4ad08ed3117ca42a3de3 [term 1 LEADER]: Raft consensus shutting down.
I20260809 02:06:43.120641 22056 raft_consensus.cc:2258] T 5af77087969c43aca136b6609222eecd P 47a423cc1a8f4ad08ed3117ca42a3de3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:06:43.122404 22056 ts_tablet_manager.cc:1905] T 5af77087969c43aca136b6609222eecd P 47a423cc1a8f4ad08ed3117ca42a3de3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:06:43.122427 22055 ts_tablet_manager.cc:1918] T 5af77087969c43aca136b6609222eecd P 7dce7e884fe2443781c4b797f720064e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:06:43.122658 22055 log.cc:1196] T 5af77087969c43aca136b6609222eecd P 7dce7e884fe2443781c4b797f720064e: Deleting WAL directory at /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786240941968412-1795-0/minicluster-data/ts-1/wal/wals/5af77087969c43aca136b6609222eecd
I20260809 02:06:43.123137 22055 ts_tablet_manager.cc:1939] T 5af77087969c43aca136b6609222eecd P 7dce7e884fe2443781c4b797f720064e: Deleting consensus metadata
I20260809 02:06:43.124593 21509 catalog_manager.cc:4841] TS 7dce7e884fe2443781c4b797f720064e (127.1.192.194:44689): tablet 5af77087969c43aca136b6609222eecd (table test-table [id=9fc4a6f4b2b44b529a8f534a45c08f62]) successfully deleted
W20260809 02:06:43.124615 21507 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.192.193:44153 (error 108)
I20260809 02:06:43.125080 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 21732
I20260809 02:06:43.127131 22056 ts_tablet_manager.cc:1918] T 5af77087969c43aca136b6609222eecd P 47a423cc1a8f4ad08ed3117ca42a3de3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:06:43.127264 22056 log.cc:1196] T 5af77087969c43aca136b6609222eecd P 47a423cc1a8f4ad08ed3117ca42a3de3: Deleting WAL directory at /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786240941968412-1795-0/minicluster-data/ts-2/wal/wals/5af77087969c43aca136b6609222eecd
I20260809 02:06:43.127676 22056 ts_tablet_manager.cc:1939] T 5af77087969c43aca136b6609222eecd P 47a423cc1a8f4ad08ed3117ca42a3de3: Deleting consensus metadata
I20260809 02:06:43.127782 22027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:43.127395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.193:44153 (local address 127.0.0.1:49322)
0809 02:06:43.127595 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0809 02:06:43.127680 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.193:44153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
W20260809 02:06:43.127997 21507 catalog_manager.cc:4584] TS 3cf2b36e4e13486da203ba04d65a3aef (127.1.192.193:44153): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5af77087969c43aca136b6609222eecd: Network error: Client connection negotiation failed: client connection to 127.1.192.193:44153: connect: Connection refused (error 111)
I20260809 02:06:43.129040 21508 catalog_manager.cc:4841] TS 47a423cc1a8f4ad08ed3117ca42a3de3 (127.1.192.195:39383): tablet 5af77087969c43aca136b6609222eecd (table test-table [id=9fc4a6f4b2b44b529a8f534a45c08f62]) successfully deleted
I20260809 02:06:43.134887 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 21875
I20260809 02:06:43.141765 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 21489
2026-08-09T02:06:43Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2575 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-09T02:06:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:06:43Z Disabled control of system clock
I20260809 02:06:43.177132 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786240941968412-1795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786240941968412-1795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786240941968412-1795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.192.254:41545
--webserver_interface=127.1.192.254
--webserver_port=0
--builtin_ntp_servers=127.1.192.212:33831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.192.254:41545
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 02:06:43.291603 22061 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:06:43.291905 22061 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:06:43.291944 22061 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:06:43.296073 22061 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:06:43.296150 22061 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:06:43.296166 22061 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:06:43.296183 22061 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:06:43.296200 22061 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:06:43.301045 22061 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:33831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786240941968412-1795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786240941968412-1795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.192.254:41545
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.192.254:41545
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.192.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22061
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786240941968412-1795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:06:43.302232 22061 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:06:43.303609 22061 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:06:43.312418 22069 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:06:43.312469 22071 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:06:43.312461 22068 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:06:43.312832 22061 server_base.cc:1034] running on GCE node
I20260809 02:06:43.313426 22061 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:06:43.314615 22061 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:06:43.315788 22061 hybrid_clock.cc:648] HybridClock initialized: now 1786241203315768 us; error 32 us; skew 500 ppm
I20260809 02:06:43.317814 22061 webserver.cc:460] Webserver started at http://127.1.192.254:41735/ using document root <none> and password file <none>
I20260809 02:06:43.318389 22061 fs_manager.cc:359] Metadata directory not provided
I20260809 02:06:43.318451 22061 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:06:43.318634 22061 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:06:43.320689 22061 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786240941968412-1795-0/minicluster-data/master-0/data/instance:
uuid: "85c14d004fca483dab331cc4e3980d0a"
format_stamp: "Formatted at 2026-08-09 02:06:43 on dist-test-slave-ss4n"
server_key: "1abf5ec224899676afc8055ed6260a6e"
server_key_iv: "bdc78f3f2d3cd2a27863915e07477855"
server_key_version: "encryptionkey@0"
I20260809 02:06:43.321255 22061 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786240941968412-1795-0/minicluster-data/master-0/wal/instance:
uuid: "85c14d004fca483dab331cc4e3980d0a"
format_stamp: "Formatted at 2026-08-09 02:06:43 on dist-test-slave-ss4n"
server_key: "1abf5ec224899676afc8055ed6260a6e"
server_key_iv: "bdc78f3f2d3cd2a27863915e07477855"
server_key_version: "encryptionkey@0"
I20260809 02:06:43.325012 22061 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260809 02:06:43.327486 22077 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:06:43.328722 22061 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 02:06:43.328850 22061 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786240941968412-1795-0/minicluster-data/master-0/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786240941968412-1795-0/minicluster-data/master-0/wal
uuid: "85c14d004fca483dab331cc4e3980d0a"
format_stamp: "Formatted at 2026-08-09 02:06:43 on dist-test-slave-ss4n"
server_key: "1abf5ec224899676afc8055ed6260a6e"
server_key_iv: "bdc78f3f2d3cd2a27863915e07477855"
server_key_version: "encryptionkey@0"
I20260809 02:06:43.328964 22061 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786240941968412-1795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786240941968412-1795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786240941968412-1795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:06:43.348364 22061 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:06:43.349133 22061 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:06:43.349357 22061 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:06:43.356711 22061 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.254:41545
I20260809 02:06:43.356732 22131 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.254:41545 every 8 connection(s)
I20260809 02:06:43.357780 22061 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
I20260809 02:06:43.361018 22132 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:06:43.363762 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 22061
I20260809 02:06:43.363883 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786240941968412-1795-0/minicluster-data/master-0/wal/instance
I20260809 02:06:43.364203 1795 external_mini_cluster.cc:1428] Setting key 309574e80ea3bc5c85e22f74fc0c2044
I20260809 02:06:43.367412 22132 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 85c14d004fca483dab331cc4e3980d0a: Bootstrap starting.
I20260809 02:06:43.370255 22132 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 85c14d004fca483dab331cc4e3980d0a: Neither blocks nor log segments found. Creating new log.
I20260809 02:06:43.371099 22132 log.cc:824] T 00000000000000000000000000000000 P 85c14d004fca483dab331cc4e3980d0a: Log is configured to *not* fsync() on all Append() calls
I20260809 02:06:43.373351 22132 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 85c14d004fca483dab331cc4e3980d0a: No bootstrap required, opened a new log
I20260809 02:06:43.375202 22135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:43.365983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53204 (local address 127.1.192.254:41545)
0809 02:06:43.366731 (+ 748us) server_negotiation.cc:206] Beginning negotiation
0809 02:06:43.366754 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0809 02:06:43.366799 (+ 45us) server_negotiation.cc:407] Connection header received
0809 02:06:43.367638 (+ 839us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:06:43.367662 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:06:43.368166 (+ 504us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:06:43.368536 (+ 370us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:06:43.369026 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:43.370513 (+ 1487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:43.371505 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:43.372050 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:43.372222 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:43.373084 (+ 862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:06:43.373090 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:06:43.373107 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:06:43.373648 (+ 541us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:06:43.373721 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:06:43.373750 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:06:43.373853 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0809 02:06:43.373994 (+ 141us) server_negotiation.cc:299] Negotiation successful
0809 02:06:43.374275 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":130,"threads_started":1}
I20260809 02:06:43.377877 22132 raft_consensus.cc:348] T 00000000000000000000000000000000 P 85c14d004fca483dab331cc4e3980d0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85c14d004fca483dab331cc4e3980d0a" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 41545 } }
I20260809 02:06:43.378103 22132 raft_consensus.cc:374] T 00000000000000000000000000000000 P 85c14d004fca483dab331cc4e3980d0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:06:43.378137 22132 raft_consensus.cc:729] T 00000000000000000000000000000000 P 85c14d004fca483dab331cc4e3980d0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85c14d004fca483dab331cc4e3980d0a, State: Initialized, Role: FOLLOWER
I20260809 02:06:43.378527 22132 consensus_queue.cc:260] T 00000000000000000000000000000000 P 85c14d004fca483dab331cc4e3980d0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85c14d004fca483dab331cc4e3980d0a" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 41545 } }
I20260809 02:06:43.378654 22132 raft_consensus.cc:388] T 00000000000000000000000000000000 P 85c14d004fca483dab331cc4e3980d0a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:06:43.378701 22132 raft_consensus.cc:482] T 00000000000000000000000000000000 P 85c14d004fca483dab331cc4e3980d0a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:06:43.378774 22132 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 85c14d004fca483dab331cc4e3980d0a [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:06:43.379703 22132 raft_consensus.cc:504] T 00000000000000000000000000000000 P 85c14d004fca483dab331cc4e3980d0a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85c14d004fca483dab331cc4e3980d0a" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 41545 } }
I20260809 02:06:43.379992 22132 leader_election.cc:302] T 00000000000000000000000000000000 P 85c14d004fca483dab331cc4e3980d0a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 85c14d004fca483dab331cc4e3980d0a; no voters:
I20260809 02:06:43.380270 22132 leader_election.cc:288] T 00000000000000000000000000000000 P 85c14d004fca483dab331cc4e3980d0a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:06:43.380393 22137 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 85c14d004fca483dab331cc4e3980d0a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:06:43.380682 22137 raft_consensus.cc:686] T 00000000000000000000000000000000 P 85c14d004fca483dab331cc4e3980d0a [term 1 LEADER]: Becoming Leader. State: Replica: 85c14d004fca483dab331cc4e3980d0a, State: Running, Role: LEADER
I20260809 02:06:43.380987 22137 consensus_queue.cc:237] T 00000000000000000000000000000000 P 85c14d004fca483dab331cc4e3980d0a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85c14d004fca483dab331cc4e3980d0a" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 41545 } }
I20260809 02:06:43.381654 22132 sys_catalog.cc:564] T 00000000000000000000000000000000 P 85c14d004fca483dab331cc4e3980d0a [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:06:43.382768 22138 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85c14d004fca483dab331cc4e3980d0a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "85c14d004fca483dab331cc4e3980d0a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85c14d004fca483dab331cc4e3980d0a" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 41545 } } }
I20260809 02:06:43.382910 22138 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85c14d004fca483dab331cc4e3980d0a [sys.catalog]: This master's current role is: LEADER
I20260809 02:06:43.382768 22139 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85c14d004fca483dab331cc4e3980d0a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 85c14d004fca483dab331cc4e3980d0a. Latest consensus state: current_term: 1 leader_uuid: "85c14d004fca483dab331cc4e3980d0a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85c14d004fca483dab331cc4e3980d0a" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 41545 } } }
I20260809 02:06:43.383222 22139 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85c14d004fca483dab331cc4e3980d0a [sys.catalog]: This master's current role is: LEADER
I20260809 02:06:43.383531 22146 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:06:43.386457 22146 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:06:43.392740 22146 catalog_manager.cc:1347] Generated new cluster ID: 88db0eb0ceb24904bd9d54c88810fc2e
I20260809 02:06:43.392839 22146 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:06:43.418615 22146 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:06:43.419606 22146 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:06:43.431715 22146 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 85c14d004fca483dab331cc4e3980d0a: Generated new TSK 0
I20260809 02:06:43.432596 22146 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:06:43.438362 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786240941968412-1795-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786240941968412-1795-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786240941968412-1795-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786240941968412-1795-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.192.193:0
--local_ip_for_outbound_sockets=127.1.192.193
--webserver_interface=127.1.192.193
--webserver_port=0
--tserver_master_addrs=127.1.192.254:41545
--builtin_ntp_servers=127.1.192.212:33831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 02:06:43.560261 22158 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:06:43.560542 22158 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:06:43.560582 22158 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:06:43.564814 22158 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:06:43.564872 22158 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:06:43.564929 22158 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.192.193
I20260809 02:06:43.569880 22158 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:33831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786240941968412-1795-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786240941968412-1795-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.192.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786240941968412-1795-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.192.193
--webserver_port=0
--tserver_master_addrs=127.1.192.254:41545
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22158
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.192.193
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786240941968412-1795-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:06:43.570930 22158 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:06:43.572254 22158 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:06:43.581686 22164 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:06:43.581718 22163 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:06:43.581898 22166 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:06:43.582065 22158 server_base.cc:1034] running on GCE node
I20260809 02:06:43.582468 22158 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:06:43.583220 22158 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:06:43.584403 22158 hybrid_clock.cc:648] HybridClock initialized: now 1786241203584371 us; error 51 us; skew 500 ppm
I20260809 02:06:43.586269 22158 webserver.cc:460] Webserver started at http://127.1.192.193:35475/ using document root <none> and password file <none>
I20260809 02:06:43.586838 22158 fs_manager.cc:359] Metadata directory not provided
I20260809 02:06:43.586901 22158 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:06:43.587126 22158 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:06:43.589156 22158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786240941968412-1795-0/minicluster-data/ts-0/data/instance:
uuid: "68fc9737ae65432db2124ea0ba869530"
format_stamp: "Formatted at 2026-08-09 02:06:43 on dist-test-slave-ss4n"
server_key: "dc5d6f0ffe362890c6cd9d5b75efedd4"
server_key_iv: "249f0d79c8dd9f6f12f1d7ff68d46534"
server_key_version: "encryptionkey@0"
I20260809 02:06:43.589737 22158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786240941968412-1795-0/minicluster-data/ts-0/wal/instance:
uuid: "68fc9737ae65432db2124ea0ba869530"
format_stamp: "Formatted at 2026-08-09 02:06:43 on dist-test-slave-ss4n"
server_key: "dc5d6f0ffe362890c6cd9d5b75efedd4"
server_key_iv: "249f0d79c8dd9f6f12f1d7ff68d46534"
server_key_version: "encryptionkey@0"
I20260809 02:06:43.594023 22158 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260809 02:06:43.596530 22174 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:06:43.597889 22158 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 02:06:43.598037 22158 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786240941968412-1795-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786240941968412-1795-0/minicluster-data/ts-0/wal
uuid: "68fc9737ae65432db2124ea0ba869530"
format_stamp: "Formatted at 2026-08-09 02:06:43 on dist-test-slave-ss4n"
server_key: "dc5d6f0ffe362890c6cd9d5b75efedd4"
server_key_iv: "249f0d79c8dd9f6f12f1d7ff68d46534"
server_key_version: "encryptionkey@0"
I20260809 02:06:43.598165 22158 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786240941968412-1795-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786240941968412-1795-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786240941968412-1795-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:06:43.609717 22158 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:06:43.610410 22158 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:06:43.610612 22158 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:06:43.611219 22158 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:06:43.612183 22158 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:06:43.612251 22158 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:06:43.612320 22158 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:06:43.612360 22158 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:06:43.621309 22158 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.193:41733
I20260809 02:06:43.621369 22295 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.193:41733 every 8 connection(s)
I20260809 02:06:43.622292 22158 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786240941968412-1795-0/minicluster-data/ts-0/data/info.pb
I20260809 02:06:43.625968 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 22158
I20260809 02:06:43.626127 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786240941968412-1795-0/minicluster-data/ts-0/wal/instance
I20260809 02:06:43.626467 1795 external_mini_cluster.cc:1428] Setting key f6774525d41c02baece7b7715fc5c7fe
I20260809 02:06:43.628665 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786240941968412-1795-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786240941968412-1795-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786240941968412-1795-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786240941968412-1795-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.192.194:0
--local_ip_for_outbound_sockets=127.1.192.194
--webserver_interface=127.1.192.194
--webserver_port=0
--tserver_master_addrs=127.1.192.254:41545
--builtin_ntp_servers=127.1.192.212:33831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 02:06:43.632789 22135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:43.624228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.193:60563 (local address 127.1.192.254:41545)
0809 02:06:43.624376 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0809 02:06:43.624380 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:06:43.625474 (+ 1094us) server_negotiation.cc:407] Connection header received
0809 02:06:43.627143 (+ 1669us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:06:43.627146 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:06:43.627206 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:06:43.627311 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:06:43.628667 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:43.629286 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:43.630639 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:43.630864 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:43.630960 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:43.632320 (+ 1360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:06:43.632323 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:06:43.632326 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:06:43.632340 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:06:43.632379 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:06:43.632388 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:06:43.632446 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 02:06:43.632615 (+ 169us) server_negotiation.cc:299] Negotiation successful
0809 02:06:43.632674 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 02:06:43.634123 22298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:43.624659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:41545 (local address 127.1.192.193:60563)
0809 02:06:43.625324 (+ 665us) negotiation.cc:107] Waiting for socket to connect
0809 02:06:43.625364 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 02:06:43.626886 (+ 1522us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:06:43.627950 (+ 1064us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:06:43.627957 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:06:43.628041 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:06:43.628524 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:06:43.628535 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:06:43.629421 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:06:43.629424 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:06:43.630485 (+ 1061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:06:43.630501 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:06:43.631035 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:06:43.631041 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:06:43.631277 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:43.631998 (+ 721us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 02:06:43.632003 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:06:43.632060 (+ 57us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 02:06:43.632155 (+ 95us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:06:43.632456 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:06:43.632468 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:06:43.632477 (+ 9us) client_negotiation.cc:769] Sending connection context
0809 02:06:43.632608 (+ 131us) client_negotiation.cc:240] Negotiation successful
0809 02:06:43.632925 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":127,"threads_started":1}
I20260809 02:06:43.635645 22296 heartbeater.cc:347] Connected to a master server at 127.1.192.254:41545
I20260809 02:06:43.635906 22296 heartbeater.cc:464] Registering TS with master...
I20260809 02:06:43.636490 22296 heartbeater.cc:511] Master 127.1.192.254:41545 requested a full tablet report, sending...
I20260809 02:06:43.638186 22095 ts_manager.cc:194] Registered new tserver with Master: 68fc9737ae65432db2124ea0ba869530 (127.1.192.193:41733)
I20260809 02:06:43.639371 22095 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.192.193:60563
W20260809 02:06:43.752586 22299 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:06:43.752949 22299 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:06:43.753028 22299 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:06:43.757179 22299 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:06:43.757284 22299 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:06:43.757385 22299 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.192.194
I20260809 02:06:43.763779 22299 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:33831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786240941968412-1795-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786240941968412-1795-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.192.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786240941968412-1795-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.192.194
--webserver_port=0
--tserver_master_addrs=127.1.192.254:41545
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22299
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.192.194
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786240941968412-1795-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:06:43.764974 22299 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:06:43.766460 22299 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:06:43.776419 22307 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:06:43.776419 22304 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:06:43.776504 22305 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:06:43.776784 22299 server_base.cc:1034] running on GCE node
I20260809 02:06:43.777230 22299 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:06:43.777912 22299 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:06:43.779125 22299 hybrid_clock.cc:648] HybridClock initialized: now 1786241203779098 us; error 46 us; skew 500 ppm
I20260809 02:06:43.781127 22299 webserver.cc:460] Webserver started at http://127.1.192.194:40659/ using document root <none> and password file <none>
I20260809 02:06:43.781984 22299 fs_manager.cc:359] Metadata directory not provided
I20260809 02:06:43.782099 22299 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:06:43.782388 22299 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:06:43.785029 22299 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786240941968412-1795-0/minicluster-data/ts-1/data/instance:
uuid: "3d563744e8214676a11ac118a9f2dd27"
format_stamp: "Formatted at 2026-08-09 02:06:43 on dist-test-slave-ss4n"
server_key: "f9e0a9052ae74438ef55d0020acb7a1f"
server_key_iv: "5733db356edc24a4283999371f4603be"
server_key_version: "encryptionkey@0"
I20260809 02:06:43.785671 22299 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786240941968412-1795-0/minicluster-data/ts-1/wal/instance:
uuid: "3d563744e8214676a11ac118a9f2dd27"
format_stamp: "Formatted at 2026-08-09 02:06:43 on dist-test-slave-ss4n"
server_key: "f9e0a9052ae74438ef55d0020acb7a1f"
server_key_iv: "5733db356edc24a4283999371f4603be"
server_key_version: "encryptionkey@0"
I20260809 02:06:43.790069 22299 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260809 02:06:43.793114 22313 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:06:43.794437 22299 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 02:06:43.794590 22299 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786240941968412-1795-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786240941968412-1795-0/minicluster-data/ts-1/wal
uuid: "3d563744e8214676a11ac118a9f2dd27"
format_stamp: "Formatted at 2026-08-09 02:06:43 on dist-test-slave-ss4n"
server_key: "f9e0a9052ae74438ef55d0020acb7a1f"
server_key_iv: "5733db356edc24a4283999371f4603be"
server_key_version: "encryptionkey@0"
I20260809 02:06:43.794713 22299 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786240941968412-1795-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786240941968412-1795-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786240941968412-1795-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:06:43.809283 22299 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:06:43.810101 22299 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:06:43.810300 22299 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:06:43.810894 22299 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:06:43.811832 22299 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:06:43.811903 22299 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:06:43.811973 22299 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:06:43.812019 22299 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:06:43.820778 22299 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.194:46369
I20260809 02:06:43.820859 22431 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.194:46369 every 8 connection(s)
I20260809 02:06:43.821741 22299 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786240941968412-1795-0/minicluster-data/ts-1/data/info.pb
I20260809 02:06:43.827746 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 22299
I20260809 02:06:43.827898 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786240941968412-1795-0/minicluster-data/ts-1/wal/instance
I20260809 02:06:43.828292 1795 external_mini_cluster.cc:1428] Setting key d3ca832f00cd6e12c57ffa2820e15035
I20260809 02:06:43.830575 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786240941968412-1795-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786240941968412-1795-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786240941968412-1795-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786240941968412-1795-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.192.195:0
--local_ip_for_outbound_sockets=127.1.192.195
--webserver_interface=127.1.192.195
--webserver_port=0
--tserver_master_addrs=127.1.192.254:41545
--builtin_ntp_servers=127.1.192.212:33831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 02:06:43.832669 22135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:43.823730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.194:46855 (local address 127.1.192.254:41545)
0809 02:06:43.823912 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 02:06:43.823917 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:06:43.825009 (+ 1092us) server_negotiation.cc:407] Connection header received
0809 02:06:43.826753 (+ 1744us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:06:43.826756 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:06:43.826811 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:06:43.826898 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:06:43.828251 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:43.828856 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:43.830316 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:43.830538 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:43.830648 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:43.832110 (+ 1462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:06:43.832114 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:06:43.832116 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:06:43.832128 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:06:43.832149 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:06:43.832157 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:06:43.832238 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 02:06:43.832420 (+ 182us) server_negotiation.cc:299] Negotiation successful
0809 02:06:43.832492 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 02:06:43.833799 22434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:43.824085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:41545 (local address 127.1.192.194:46855)
0809 02:06:43.824838 (+ 753us) negotiation.cc:107] Waiting for socket to connect
0809 02:06:43.824884 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 02:06:43.826486 (+ 1602us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:06:43.827115 (+ 629us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:06:43.827122 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:06:43.827214 (+ 92us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:06:43.828051 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:06:43.828078 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:06:43.829005 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:06:43.829008 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:06:43.830183 (+ 1175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:06:43.830194 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:06:43.830697 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:06:43.830701 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:06:43.830893 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:43.831758 (+ 865us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 02:06:43.831760 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:06:43.831804 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 02:06:43.831924 (+ 120us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:06:43.832240 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:06:43.832253 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:06:43.832261 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 02:06:43.832421 (+ 160us) client_negotiation.cc:240] Negotiation successful
0809 02:06:43.832898 (+ 477us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":119,"threads_started":1}
I20260809 02:06:43.835038 22432 heartbeater.cc:347] Connected to a master server at 127.1.192.254:41545
I20260809 02:06:43.835350 22432 heartbeater.cc:464] Registering TS with master...
I20260809 02:06:43.836118 22432 heartbeater.cc:511] Master 127.1.192.254:41545 requested a full tablet report, sending...
I20260809 02:06:43.837510 22095 ts_manager.cc:194] Registered new tserver with Master: 3d563744e8214676a11ac118a9f2dd27 (127.1.192.194:46369)
I20260809 02:06:43.838510 22095 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.192.194:46855
W20260809 02:06:43.954082 22435 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:06:43.954360 22435 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:06:43.954401 22435 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:06:43.958592 22435 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:06:43.958660 22435 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:06:43.958724 22435 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.192.195
I20260809 02:06:43.963732 22435 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:33831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786240941968412-1795-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786240941968412-1795-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.192.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786240941968412-1795-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.192.195
--webserver_port=0
--tserver_master_addrs=127.1.192.254:41545
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22435
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.192.195
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786240941968412-1795-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:06:43.964883 22435 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:06:43.966341 22435 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:06:43.976838 22443 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:06:43.976838 22440 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:06:43.976900 22441 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:06:43.977102 22435 server_base.cc:1034] running on GCE node
I20260809 02:06:43.977809 22435 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:06:43.978546 22435 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:06:43.979759 22435 hybrid_clock.cc:648] HybridClock initialized: now 1786241203979739 us; error 36 us; skew 500 ppm
I20260809 02:06:43.982311 22435 webserver.cc:460] Webserver started at http://127.1.192.195:39533/ using document root <none> and password file <none>
I20260809 02:06:43.983002 22435 fs_manager.cc:359] Metadata directory not provided
I20260809 02:06:43.983160 22435 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:06:43.983400 22435 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:06:43.985781 22435 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786240941968412-1795-0/minicluster-data/ts-2/data/instance:
uuid: "6bde50f595cc4e798618343fd17526cc"
format_stamp: "Formatted at 2026-08-09 02:06:43 on dist-test-slave-ss4n"
server_key: "7fad9ef90d529b6938bb2aa04d75cddf"
server_key_iv: "e68121723e53b1e1243bbceb37112baa"
server_key_version: "encryptionkey@0"
I20260809 02:06:43.986431 22435 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786240941968412-1795-0/minicluster-data/ts-2/wal/instance:
uuid: "6bde50f595cc4e798618343fd17526cc"
format_stamp: "Formatted at 2026-08-09 02:06:43 on dist-test-slave-ss4n"
server_key: "7fad9ef90d529b6938bb2aa04d75cddf"
server_key_iv: "e68121723e53b1e1243bbceb37112baa"
server_key_version: "encryptionkey@0"
I20260809 02:06:43.990562 22435 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 02:06:43.993440 22449 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:06:43.994943 22435 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 02:06:43.995098 22435 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786240941968412-1795-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786240941968412-1795-0/minicluster-data/ts-2/wal
uuid: "6bde50f595cc4e798618343fd17526cc"
format_stamp: "Formatted at 2026-08-09 02:06:43 on dist-test-slave-ss4n"
server_key: "7fad9ef90d529b6938bb2aa04d75cddf"
server_key_iv: "e68121723e53b1e1243bbceb37112baa"
server_key_version: "encryptionkey@0"
I20260809 02:06:43.995235 22435 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786240941968412-1795-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786240941968412-1795-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786240941968412-1795-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:06:44.012924 22435 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:06:44.013793 22435 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:06:44.014007 22435 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:06:44.014633 22435 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:06:44.015622 22435 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:06:44.015700 22435 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:06:44.015769 22435 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:06:44.015815 22435 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:06:44.024825 22435 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.195:34343
I20260809 02:06:44.024897 22568 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.195:34343 every 8 connection(s)
I20260809 02:06:44.025866 22435 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786240941968412-1795-0/minicluster-data/ts-2/data/info.pb
I20260809 02:06:44.029331 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 22435
I20260809 02:06:44.029453 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786240941968412-1795-0/minicluster-data/ts-2/wal/instance
I20260809 02:06:44.029829 1795 external_mini_cluster.cc:1428] Setting key 5587b4d32778b1431291008a675fe7f5
I20260809 02:06:44.036624 22135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:44.027866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.195:41543 (local address 127.1.192.254:41545)
0809 02:06:44.028062 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0809 02:06:44.028066 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:06:44.029296 (+ 1230us) server_negotiation.cc:407] Connection header received
0809 02:06:44.031361 (+ 2065us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:06:44.031363 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:06:44.031413 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:06:44.031479 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:06:44.032638 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:44.033199 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:44.034622 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:44.034853 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:44.034942 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:44.036173 (+ 1231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:06:44.036176 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:06:44.036178 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:06:44.036191 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:06:44.036212 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:06:44.036220 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:06:44.036282 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 02:06:44.036443 (+ 161us) server_negotiation.cc:299] Negotiation successful
0809 02:06:44.036501 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260809 02:06:44.037724 22571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:44.028233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:41545 (local address 127.1.192.195:41543)
0809 02:06:44.029094 (+ 861us) negotiation.cc:107] Waiting for socket to connect
0809 02:06:44.029158 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0809 02:06:44.031099 (+ 1941us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:06:44.031687 (+ 588us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:06:44.031694 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:06:44.031785 (+ 91us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:06:44.032490 (+ 705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:06:44.032507 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:06:44.033354 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:06:44.033357 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:06:44.034490 (+ 1133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:06:44.034500 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:06:44.035034 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:06:44.035038 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:06:44.035203 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:44.035866 (+ 663us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 02:06:44.035867 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:06:44.035912 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 02:06:44.035986 (+ 74us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:06:44.036305 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:06:44.036313 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:06:44.036318 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 02:06:44.036434 (+ 116us) client_negotiation.cc:240] Negotiation successful
0809 02:06:44.036707 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":124,"threads_started":1}
I20260809 02:06:44.038970 22569 heartbeater.cc:347] Connected to a master server at 127.1.192.254:41545
I20260809 02:06:44.039253 22569 heartbeater.cc:464] Registering TS with master...
I20260809 02:06:44.039846 22569 heartbeater.cc:511] Master 127.1.192.254:41545 requested a full tablet report, sending...
I20260809 02:06:44.041031 22095 ts_manager.cc:194] Registered new tserver with Master: 6bde50f595cc4e798618343fd17526cc (127.1.192.195:34343)
I20260809 02:06:44.041792 22095 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.192.195:41543
I20260809 02:06:44.044370 1795 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 02:06:44.050776 22135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:44.046466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53214 (local address 127.1.192.254:41545)
0809 02:06:44.046669 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0809 02:06:44.046674 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:06:44.046837 (+ 163us) server_negotiation.cc:407] Connection header received
0809 02:06:44.046985 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:06:44.046989 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:06:44.047034 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:06:44.047156 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:06:44.047446 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:44.047994 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:44.048860 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:44.049100 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:44.049211 (+ 111us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:44.050200 (+ 989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:06:44.050204 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:06:44.050206 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:06:44.050218 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:06:44.050237 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:06:44.050246 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:06:44.050383 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0809 02:06:44.050553 (+ 170us) server_negotiation.cc:299] Negotiation successful
0809 02:06:44.050597 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260809 02:06:44.054286 22095 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53214:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 02:06:44.057842 22095 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:06:44.074446 22584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:44.068464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.195:34343 (local address 127.0.0.1:60148)
0809 02:06:44.069360 (+ 896us) negotiation.cc:107] Waiting for socket to connect
0809 02:06:44.069377 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 02:06:44.069537 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:06:44.070052 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:06:44.070055 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:06:44.070077 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:06:44.070165 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:06:44.070171 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:06:44.071917 (+ 1746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:06:44.071920 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:06:44.072956 (+ 1036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:06:44.072963 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:06:44.073837 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:06:44.073841 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:06:44.073948 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:44.073996 (+ 48us) client_negotiation.cc:769] Sending connection context
0809 02:06:44.074121 (+ 125us) client_negotiation.cc:240] Negotiation successful
0809 02:06:44.074269 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":830,"mutex_wait_us":3,"thread_start_us":114,"threads_started":1}
I20260809 02:06:44.074512 22585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:44.068920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60148 (local address 127.1.192.195:34343)
0809 02:06:44.069294 (+ 374us) server_negotiation.cc:206] Beginning negotiation
0809 02:06:44.069305 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 02:06:44.069534 (+ 229us) server_negotiation.cc:407] Connection header received
0809 02:06:44.069668 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:06:44.069675 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:06:44.069853 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:06:44.070019 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:06:44.070283 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:44.071537 (+ 1254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:44.073084 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:44.073708 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:44.073836 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:44.074042 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0809 02:06:44.074171 (+ 129us) server_negotiation.cc:299] Negotiation successful
0809 02:06:44.074298 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":274,"thread_start_us":158,"threads_started":1}
I20260809 02:06:44.074843 22583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:44.067979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54278 (local address 127.1.192.193:41733)
0809 02:06:44.068636 (+ 657us) server_negotiation.cc:206] Beginning negotiation
0809 02:06:44.068648 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 02:06:44.068884 (+ 236us) server_negotiation.cc:407] Connection header received
0809 02:06:44.069144 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:06:44.069152 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:06:44.069358 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:06:44.069526 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:06:44.070034 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:44.071225 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:44.072799 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:44.073734 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:44.073854 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:44.074047 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0809 02:06:44.074501 (+ 454us) server_negotiation.cc:299] Negotiation successful
0809 02:06:44.074637 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":554,"thread_start_us":111,"threads_started":1}
I20260809 02:06:44.075011 22582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:44.067763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51748 (local address 127.1.192.194:46369)
0809 02:06:44.068417 (+ 654us) server_negotiation.cc:206] Beginning negotiation
0809 02:06:44.068431 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 02:06:44.068987 (+ 556us) server_negotiation.cc:407] Connection header received
0809 02:06:44.069118 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:06:44.069124 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:06:44.069359 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:06:44.069548 (+ 189us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:06:44.070016 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:44.070950 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:44.072784 (+ 1834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:44.073733 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:44.073854 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:44.074057 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0809 02:06:44.074723 (+ 666us) server_negotiation.cc:299] Negotiation successful
0809 02:06:44.074825 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":528,"thread_start_us":171,"threads_started":1}
I20260809 02:06:44.075296 22580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:44.067696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.193:41733 (local address 127.0.0.1:54278)
0809 02:06:44.068777 (+ 1081us) negotiation.cc:107] Waiting for socket to connect
0809 02:06:44.068801 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 02:06:44.068962 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:06:44.069726 (+ 764us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:06:44.069729 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:06:44.069747 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:06:44.069894 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:06:44.069906 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:06:44.071352 (+ 1446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:06:44.071355 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:06:44.072668 (+ 1313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:06:44.072678 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:06:44.074125 (+ 1447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:06:44.074130 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:06:44.074320 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:44.074346 (+ 26us) client_negotiation.cc:769] Sending connection context
0809 02:06:44.075060 (+ 714us) client_negotiation.cc:240] Negotiation successful
0809 02:06:44.075145 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1017,"mutex_wait_us":119,"thread_start_us":112,"threads_started":1}
I20260809 02:06:44.074628 22581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:44.067639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.194:46369 (local address 127.0.0.1:51748)
0809 02:06:44.068597 (+ 958us) negotiation.cc:107] Waiting for socket to connect
0809 02:06:44.068656 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0809 02:06:44.068819 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:06:44.069681 (+ 862us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:06:44.069688 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:06:44.069715 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:06:44.069884 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:06:44.069890 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:06:44.071081 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:06:44.071088 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:06:44.072603 (+ 1515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:06:44.072615 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:06:44.074127 (+ 1512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:06:44.074131 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:06:44.074308 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:44.074336 (+ 28us) client_negotiation.cc:769] Sending connection context
0809 02:06:44.074405 (+ 69us) client_negotiation.cc:240] Negotiation successful
0809 02:06:44.074485 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":739,"mutex_wait_us":66,"thread_start_us":93,"threads_started":1}
I20260809 02:06:44.078627 22499 tablet_service.cc:1467] Processing CreateTablet for tablet c993920252fb442793b5bfd712ce6b85 (DEFAULT_TABLE table=test-table [id=13cbabc485af439e8c7c925bcbf3fe0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:06:44.078636 22226 tablet_service.cc:1467] Processing CreateTablet for tablet c993920252fb442793b5bfd712ce6b85 (DEFAULT_TABLE table=test-table [id=13cbabc485af439e8c7c925bcbf3fe0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:06:44.078610 22363 tablet_service.cc:1467] Processing CreateTablet for tablet c993920252fb442793b5bfd712ce6b85 (DEFAULT_TABLE table=test-table [id=13cbabc485af439e8c7c925bcbf3fe0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:06:44.079815 22499 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c993920252fb442793b5bfd712ce6b85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:06:44.079815 22226 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c993920252fb442793b5bfd712ce6b85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:06:44.079823 22363 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c993920252fb442793b5bfd712ce6b85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:06:44.087091 22586 tablet_bootstrap.cc:492] T c993920252fb442793b5bfd712ce6b85 P 3d563744e8214676a11ac118a9f2dd27: Bootstrap starting.
I20260809 02:06:44.088280 22588 tablet_bootstrap.cc:492] T c993920252fb442793b5bfd712ce6b85 P 68fc9737ae65432db2124ea0ba869530: Bootstrap starting.
I20260809 02:06:44.089849 22586 tablet_bootstrap.cc:654] T c993920252fb442793b5bfd712ce6b85 P 3d563744e8214676a11ac118a9f2dd27: Neither blocks nor log segments found. Creating new log.
I20260809 02:06:44.090636 22588 tablet_bootstrap.cc:654] T c993920252fb442793b5bfd712ce6b85 P 68fc9737ae65432db2124ea0ba869530: Neither blocks nor log segments found. Creating new log.
I20260809 02:06:44.090862 22586 log.cc:824] T c993920252fb442793b5bfd712ce6b85 P 3d563744e8214676a11ac118a9f2dd27: Log is configured to *not* fsync() on all Append() calls
I20260809 02:06:44.091573 22588 log.cc:824] T c993920252fb442793b5bfd712ce6b85 P 68fc9737ae65432db2124ea0ba869530: Log is configured to *not* fsync() on all Append() calls
I20260809 02:06:44.091739 22587 tablet_bootstrap.cc:492] T c993920252fb442793b5bfd712ce6b85 P 6bde50f595cc4e798618343fd17526cc: Bootstrap starting.
I20260809 02:06:44.093269 22586 tablet_bootstrap.cc:492] T c993920252fb442793b5bfd712ce6b85 P 3d563744e8214676a11ac118a9f2dd27: No bootstrap required, opened a new log
I20260809 02:06:44.093475 22586 ts_tablet_manager.cc:1397] T c993920252fb442793b5bfd712ce6b85 P 3d563744e8214676a11ac118a9f2dd27: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260809 02:06:44.093915 22588 tablet_bootstrap.cc:492] T c993920252fb442793b5bfd712ce6b85 P 68fc9737ae65432db2124ea0ba869530: No bootstrap required, opened a new log
I20260809 02:06:44.094110 22588 ts_tablet_manager.cc:1397] T c993920252fb442793b5bfd712ce6b85 P 68fc9737ae65432db2124ea0ba869530: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260809 02:06:44.094110 22587 tablet_bootstrap.cc:654] T c993920252fb442793b5bfd712ce6b85 P 6bde50f595cc4e798618343fd17526cc: Neither blocks nor log segments found. Creating new log.
I20260809 02:06:44.095044 22587 log.cc:824] T c993920252fb442793b5bfd712ce6b85 P 6bde50f595cc4e798618343fd17526cc: Log is configured to *not* fsync() on all Append() calls
I20260809 02:06:44.097193 22587 tablet_bootstrap.cc:492] T c993920252fb442793b5bfd712ce6b85 P 6bde50f595cc4e798618343fd17526cc: No bootstrap required, opened a new log
I20260809 02:06:44.097386 22587 ts_tablet_manager.cc:1397] T c993920252fb442793b5bfd712ce6b85 P 6bde50f595cc4e798618343fd17526cc: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260809 02:06:44.098464 22586 raft_consensus.cc:348] T c993920252fb442793b5bfd712ce6b85 P 3d563744e8214676a11ac118a9f2dd27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d563744e8214676a11ac118a9f2dd27" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 46369 } } peers { permanent_uuid: "68fc9737ae65432db2124ea0ba869530" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 41733 } } peers { permanent_uuid: "6bde50f595cc4e798618343fd17526cc" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 34343 } }
I20260809 02:06:44.098721 22586 raft_consensus.cc:374] T c993920252fb442793b5bfd712ce6b85 P 3d563744e8214676a11ac118a9f2dd27 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:06:44.098817 22586 raft_consensus.cc:729] T c993920252fb442793b5bfd712ce6b85 P 3d563744e8214676a11ac118a9f2dd27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d563744e8214676a11ac118a9f2dd27, State: Initialized, Role: FOLLOWER
I20260809 02:06:44.099291 22586 consensus_queue.cc:260] T c993920252fb442793b5bfd712ce6b85 P 3d563744e8214676a11ac118a9f2dd27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d563744e8214676a11ac118a9f2dd27" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 46369 } } peers { permanent_uuid: "68fc9737ae65432db2124ea0ba869530" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 41733 } } peers { permanent_uuid: "6bde50f595cc4e798618343fd17526cc" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 34343 } }
I20260809 02:06:44.099728 22588 raft_consensus.cc:348] T c993920252fb442793b5bfd712ce6b85 P 68fc9737ae65432db2124ea0ba869530 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d563744e8214676a11ac118a9f2dd27" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 46369 } } peers { permanent_uuid: "68fc9737ae65432db2124ea0ba869530" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 41733 } } peers { permanent_uuid: "6bde50f595cc4e798618343fd17526cc" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 34343 } }
I20260809 02:06:44.099990 22588 raft_consensus.cc:374] T c993920252fb442793b5bfd712ce6b85 P 68fc9737ae65432db2124ea0ba869530 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:06:44.100057 22588 raft_consensus.cc:729] T c993920252fb442793b5bfd712ce6b85 P 68fc9737ae65432db2124ea0ba869530 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68fc9737ae65432db2124ea0ba869530, State: Initialized, Role: FOLLOWER
I20260809 02:06:44.100384 22586 ts_tablet_manager.cc:1428] T c993920252fb442793b5bfd712ce6b85 P 3d563744e8214676a11ac118a9f2dd27: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260809 02:06:44.100615 22588 consensus_queue.cc:260] T c993920252fb442793b5bfd712ce6b85 P 68fc9737ae65432db2124ea0ba869530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d563744e8214676a11ac118a9f2dd27" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 46369 } } peers { permanent_uuid: "68fc9737ae65432db2124ea0ba869530" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 41733 } } peers { permanent_uuid: "6bde50f595cc4e798618343fd17526cc" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 34343 } }
I20260809 02:06:44.100795 22432 heartbeater.cc:503] Master 127.1.192.254:41545 was elected leader, sending a full tablet report...
I20260809 02:06:44.102550 22588 ts_tablet_manager.cc:1428] T c993920252fb442793b5bfd712ce6b85 P 68fc9737ae65432db2124ea0ba869530: Time spent starting tablet: real 0.008s user 0.008s sys 0.000s
I20260809 02:06:44.105422 22296 heartbeater.cc:503] Master 127.1.192.254:41545 was elected leader, sending a full tablet report...
I20260809 02:06:44.107368 22587 raft_consensus.cc:348] T c993920252fb442793b5bfd712ce6b85 P 6bde50f595cc4e798618343fd17526cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d563744e8214676a11ac118a9f2dd27" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 46369 } } peers { permanent_uuid: "68fc9737ae65432db2124ea0ba869530" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 41733 } } peers { permanent_uuid: "6bde50f595cc4e798618343fd17526cc" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 34343 } }
I20260809 02:06:44.107702 22587 raft_consensus.cc:374] T c993920252fb442793b5bfd712ce6b85 P 6bde50f595cc4e798618343fd17526cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:06:44.107776 22587 raft_consensus.cc:729] T c993920252fb442793b5bfd712ce6b85 P 6bde50f595cc4e798618343fd17526cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bde50f595cc4e798618343fd17526cc, State: Initialized, Role: FOLLOWER
I20260809 02:06:44.108233 22587 consensus_queue.cc:260] T c993920252fb442793b5bfd712ce6b85 P 6bde50f595cc4e798618343fd17526cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d563744e8214676a11ac118a9f2dd27" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 46369 } } peers { permanent_uuid: "68fc9737ae65432db2124ea0ba869530" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 41733 } } peers { permanent_uuid: "6bde50f595cc4e798618343fd17526cc" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 34343 } }
I20260809 02:06:44.109011 22569 heartbeater.cc:503] Master 127.1.192.254:41545 was elected leader, sending a full tablet report...
I20260809 02:06:44.109417 22587 ts_tablet_manager.cc:1428] T c993920252fb442793b5bfd712ce6b85 P 6bde50f595cc4e798618343fd17526cc: Time spent starting tablet: real 0.012s user 0.008s sys 0.000s
W20260809 02:06:44.123534 22297 tablet.cc:2369] T c993920252fb442793b5bfd712ce6b85 P 68fc9737ae65432db2124ea0ba869530: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 02:06:44.276944 22570 tablet.cc:2369] T c993920252fb442793b5bfd712ce6b85 P 6bde50f595cc4e798618343fd17526cc: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 02:06:44.323025 22433 tablet.cc:2369] T c993920252fb442793b5bfd712ce6b85 P 3d563744e8214676a11ac118a9f2dd27: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 02:06:44.379511 22593 raft_consensus.cc:482] T c993920252fb442793b5bfd712ce6b85 P 68fc9737ae65432db2124ea0ba869530 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:06:44.379739 22593 raft_consensus.cc:504] T c993920252fb442793b5bfd712ce6b85 P 68fc9737ae65432db2124ea0ba869530 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d563744e8214676a11ac118a9f2dd27" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 46369 } } peers { permanent_uuid: "68fc9737ae65432db2124ea0ba869530" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 41733 } } peers { permanent_uuid: "6bde50f595cc4e798618343fd17526cc" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 34343 } }
I20260809 02:06:44.381038 22593 leader_election.cc:288] T c993920252fb442793b5bfd712ce6b85 P 68fc9737ae65432db2124ea0ba869530 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3d563744e8214676a11ac118a9f2dd27 (127.1.192.194:46369), 6bde50f595cc4e798618343fd17526cc (127.1.192.195:34343)
I20260809 02:06:44.385378 22582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:44.381457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.193:51973 (local address 127.1.192.194:46369)
0809 02:06:44.381766 (+ 309us) server_negotiation.cc:206] Beginning negotiation
0809 02:06:44.381773 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 02:06:44.381991 (+ 218us) server_negotiation.cc:407] Connection header received
0809 02:06:44.382149 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:06:44.382155 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:06:44.382219 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:06:44.382301 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:06:44.382577 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:44.383181 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:44.384385 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:44.384792 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:44.384919 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:44.384986 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 02:06:44.385173 (+ 187us) server_negotiation.cc:299] Negotiation successful
0809 02:06:44.385242 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":193}
I20260809 02:06:44.385406 22596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:44.381350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.194:46369 (local address 127.1.192.193:51973)
0809 02:06:44.381876 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0809 02:06:44.381886 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 02:06:44.382012 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:06:44.382324 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:06:44.382327 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:06:44.382343 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:06:44.382470 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:06:44.382475 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:06:44.383282 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:06:44.383288 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:06:44.384259 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:06:44.384269 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:06:44.384917 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:06:44.384921 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:06:44.385044 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:44.385064 (+ 20us) client_negotiation.cc:769] Sending connection context
0809 02:06:44.385143 (+ 79us) client_negotiation.cc:240] Negotiation successful
0809 02:06:44.385220 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":471,"mutex_wait_us":186,"thread_start_us":92,"threads_started":1}
I20260809 02:06:44.385432 22585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:44.381621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.193:57775 (local address 127.1.192.195:34343)
0809 02:06:44.381758 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0809 02:06:44.381766 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 02:06:44.381814 (+ 48us) server_negotiation.cc:407] Connection header received
0809 02:06:44.382006 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:06:44.382009 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:06:44.382058 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:06:44.382185 (+ 127us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:06:44.382533 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:44.383195 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:44.384383 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:44.384831 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:44.384967 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:44.385017 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 02:06:44.385218 (+ 201us) server_negotiation.cc:299] Negotiation successful
0809 02:06:44.385293 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 02:06:44.385882 22595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:44.381350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.195:34343 (local address 127.1.192.193:57775)
0809 02:06:44.381718 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0809 02:06:44.381735 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 02:06:44.381881 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:06:44.382202 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:06:44.382205 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:06:44.382243 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:06:44.382410 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:06:44.382417 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:06:44.383305 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:06:44.383310 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:06:44.384254 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:06:44.384263 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:06:44.384962 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:06:44.384966 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:06:44.385090 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:44.385108 (+ 18us) client_negotiation.cc:769] Sending connection context
0809 02:06:44.385200 (+ 92us) client_negotiation.cc:240] Negotiation successful
0809 02:06:44.385276 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"mutex_wait_us":22,"thread_start_us":122,"threads_started":1}
I20260809 02:06:44.386029 22384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c993920252fb442793b5bfd712ce6b85" candidate_uuid: "68fc9737ae65432db2124ea0ba869530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d563744e8214676a11ac118a9f2dd27" is_pre_election: true
I20260809 02:06:44.386385 22384 raft_consensus.cc:2454] T c993920252fb442793b5bfd712ce6b85 P 3d563744e8214676a11ac118a9f2dd27 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68fc9737ae65432db2124ea0ba869530 in term 0.
I20260809 02:06:44.386579 22520 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c993920252fb442793b5bfd712ce6b85" candidate_uuid: "68fc9737ae65432db2124ea0ba869530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bde50f595cc4e798618343fd17526cc" is_pre_election: true
I20260809 02:06:44.386857 22520 raft_consensus.cc:2454] T c993920252fb442793b5bfd712ce6b85 P 6bde50f595cc4e798618343fd17526cc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68fc9737ae65432db2124ea0ba869530 in term 0.
I20260809 02:06:44.386878 22177 leader_election.cc:302] T c993920252fb442793b5bfd712ce6b85 P 68fc9737ae65432db2124ea0ba869530 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d563744e8214676a11ac118a9f2dd27, 68fc9737ae65432db2124ea0ba869530; no voters:
I20260809 02:06:44.387172 22593 raft_consensus.cc:2781] T c993920252fb442793b5bfd712ce6b85 P 68fc9737ae65432db2124ea0ba869530 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:06:44.387303 22593 raft_consensus.cc:482] T c993920252fb442793b5bfd712ce6b85 P 68fc9737ae65432db2124ea0ba869530 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:06:44.387365 22593 raft_consensus.cc:3037] T c993920252fb442793b5bfd712ce6b85 P 68fc9737ae65432db2124ea0ba869530 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:06:44.388485 22593 raft_consensus.cc:504] T c993920252fb442793b5bfd712ce6b85 P 68fc9737ae65432db2124ea0ba869530 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d563744e8214676a11ac118a9f2dd27" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 46369 } } peers { permanent_uuid: "68fc9737ae65432db2124ea0ba869530" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 41733 } } peers { permanent_uuid: "6bde50f595cc4e798618343fd17526cc" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 34343 } }
I20260809 02:06:44.388854 22593 leader_election.cc:288] T c993920252fb442793b5bfd712ce6b85 P 68fc9737ae65432db2124ea0ba869530 [CANDIDATE]: Term 1 election: Requested vote from peers 3d563744e8214676a11ac118a9f2dd27 (127.1.192.194:46369), 6bde50f595cc4e798618343fd17526cc (127.1.192.195:34343)
I20260809 02:06:44.389231 22520 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c993920252fb442793b5bfd712ce6b85" candidate_uuid: "68fc9737ae65432db2124ea0ba869530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bde50f595cc4e798618343fd17526cc"
I20260809 02:06:44.389231 22384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c993920252fb442793b5bfd712ce6b85" candidate_uuid: "68fc9737ae65432db2124ea0ba869530" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d563744e8214676a11ac118a9f2dd27"
I20260809 02:06:44.389365 22520 raft_consensus.cc:3037] T c993920252fb442793b5bfd712ce6b85 P 6bde50f595cc4e798618343fd17526cc [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:06:44.389365 22384 raft_consensus.cc:3037] T c993920252fb442793b5bfd712ce6b85 P 3d563744e8214676a11ac118a9f2dd27 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:06:44.390826 22384 raft_consensus.cc:2454] T c993920252fb442793b5bfd712ce6b85 P 3d563744e8214676a11ac118a9f2dd27 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68fc9737ae65432db2124ea0ba869530 in term 1.
I20260809 02:06:44.390826 22520 raft_consensus.cc:2454] T c993920252fb442793b5bfd712ce6b85 P 6bde50f595cc4e798618343fd17526cc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68fc9737ae65432db2124ea0ba869530 in term 1.
I20260809 02:06:44.391173 22177 leader_election.cc:302] T c993920252fb442793b5bfd712ce6b85 P 68fc9737ae65432db2124ea0ba869530 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d563744e8214676a11ac118a9f2dd27, 68fc9737ae65432db2124ea0ba869530; no voters:
I20260809 02:06:44.391376 22593 raft_consensus.cc:2781] T c993920252fb442793b5bfd712ce6b85 P 68fc9737ae65432db2124ea0ba869530 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:06:44.391707 22593 raft_consensus.cc:686] T c993920252fb442793b5bfd712ce6b85 P 68fc9737ae65432db2124ea0ba869530 [term 1 LEADER]: Becoming Leader. State: Replica: 68fc9737ae65432db2124ea0ba869530, State: Running, Role: LEADER
I20260809 02:06:44.392113 22593 consensus_queue.cc:237] T c993920252fb442793b5bfd712ce6b85 P 68fc9737ae65432db2124ea0ba869530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d563744e8214676a11ac118a9f2dd27" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 46369 } } peers { permanent_uuid: "68fc9737ae65432db2124ea0ba869530" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 41733 } } peers { permanent_uuid: "6bde50f595cc4e798618343fd17526cc" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 34343 } }
I20260809 02:06:44.395711 22094 catalog_manager.cc:5495] T c993920252fb442793b5bfd712ce6b85 P 68fc9737ae65432db2124ea0ba869530 reported cstate change: term changed from 0 to 1, leader changed from <none> to 68fc9737ae65432db2124ea0ba869530 (127.1.192.193). New cstate: current_term: 1 leader_uuid: "68fc9737ae65432db2124ea0ba869530" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d563744e8214676a11ac118a9f2dd27" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 46369 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "68fc9737ae65432db2124ea0ba869530" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 41733 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6bde50f595cc4e798618343fd17526cc" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 34343 } health_report { overall_health: UNKNOWN } } }
I20260809 02:06:44.449221 22583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:44.445326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54282 (local address 127.1.192.193:41733)
0809 02:06:44.445475 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0809 02:06:44.445507 (+ 32us) server_negotiation.cc:399] Waiting for connection header
0809 02:06:44.445524 (+ 17us) server_negotiation.cc:407] Connection header received
0809 02:06:44.445793 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:06:44.445796 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:06:44.445867 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:06:44.445996 (+ 129us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:06:44.446263 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:44.446778 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:44.447644 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:44.447864 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:44.447975 (+ 111us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:44.448303 (+ 328us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:06:44.448717 (+ 414us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:06:44.448873 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0809 02:06:44.448979 (+ 106us) server_negotiation.cc:299] Negotiation successful
0809 02:06:44.449025 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 02:06:44.455538 22384 raft_consensus.cc:1261] T c993920252fb442793b5bfd712ce6b85 P 3d563744e8214676a11ac118a9f2dd27 [term 1 FOLLOWER]: Refusing update from remote peer 68fc9737ae65432db2124ea0ba869530: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:06:44.455538 22520 raft_consensus.cc:1261] T c993920252fb442793b5bfd712ce6b85 P 6bde50f595cc4e798618343fd17526cc [term 1 FOLLOWER]: Refusing update from remote peer 68fc9737ae65432db2124ea0ba869530: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:06:44.456328 22597 consensus_queue.cc:1035] T c993920252fb442793b5bfd712ce6b85 P 68fc9737ae65432db2124ea0ba869530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6bde50f595cc4e798618343fd17526cc" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 34343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:06:44.456548 22593 consensus_queue.cc:1035] T c993920252fb442793b5bfd712ce6b85 P 68fc9737ae65432db2124ea0ba869530 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d563744e8214676a11ac118a9f2dd27" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 46369 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:06:44.477345 22090 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:53214:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:06:44.477589 22090 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:53214:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:06:44.480417 22090 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 85c14d004fca483dab331cc4e3980d0a: Sending DeleteTablet for 3 replicas of tablet c993920252fb442793b5bfd712ce6b85
I20260809 02:06:44.481372 22226 tablet_service.cc:1514] Processing DeleteTablet for tablet c993920252fb442793b5bfd712ce6b85 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:06:44 UTC) from {username='slave'} at 127.0.0.1:54278
I20260809 02:06:44.481416 22499 tablet_service.cc:1514] Processing DeleteTablet for tablet c993920252fb442793b5bfd712ce6b85 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:06:44 UTC) from {username='slave'} at 127.0.0.1:60148
I20260809 02:06:44.481379 22363 tablet_service.cc:1514] Processing DeleteTablet for tablet c993920252fb442793b5bfd712ce6b85 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:06:44 UTC) from {username='slave'} at 127.0.0.1:51748
I20260809 02:06:44.481981 22611 tablet_replica.cc:330] T c993920252fb442793b5bfd712ce6b85 P 68fc9737ae65432db2124ea0ba869530: stopping tablet replica
I20260809 02:06:44.481983 22610 tablet_replica.cc:330] T c993920252fb442793b5bfd712ce6b85 P 6bde50f595cc4e798618343fd17526cc: stopping tablet replica
I20260809 02:06:44.482432 22611 raft_consensus.cc:2229] T c993920252fb442793b5bfd712ce6b85 P 68fc9737ae65432db2124ea0ba869530 [term 1 LEADER]: Raft consensus shutting down.
I20260809 02:06:44.482545 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 22158
I20260809 02:06:44.485379 22610 raft_consensus.cc:2229] T c993920252fb442793b5bfd712ce6b85 P 6bde50f595cc4e798618343fd17526cc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:06:44.485764 22610 raft_consensus.cc:2258] T c993920252fb442793b5bfd712ce6b85 P 6bde50f595cc4e798618343fd17526cc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:06:44.487182 22612 tablet_replica.cc:330] T c993920252fb442793b5bfd712ce6b85 P 3d563744e8214676a11ac118a9f2dd27: stopping tablet replica
I20260809 02:06:44.487490 22612 raft_consensus.cc:2229] T c993920252fb442793b5bfd712ce6b85 P 3d563744e8214676a11ac118a9f2dd27 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:06:44.487802 22612 raft_consensus.cc:2258] T c993920252fb442793b5bfd712ce6b85 P 3d563744e8214676a11ac118a9f2dd27 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:06:44.487799 22610 ts_tablet_manager.cc:1905] T c993920252fb442793b5bfd712ce6b85 P 6bde50f595cc4e798618343fd17526cc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:06:44.489756 22612 ts_tablet_manager.cc:1905] T c993920252fb442793b5bfd712ce6b85 P 3d563744e8214676a11ac118a9f2dd27: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:06:44.491338 22610 ts_tablet_manager.cc:1918] T c993920252fb442793b5bfd712ce6b85 P 6bde50f595cc4e798618343fd17526cc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:06:44.491443 22610 log.cc:1196] T c993920252fb442793b5bfd712ce6b85 P 6bde50f595cc4e798618343fd17526cc: Deleting WAL directory at /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786240941968412-1795-0/minicluster-data/ts-2/wal/wals/c993920252fb442793b5bfd712ce6b85
I20260809 02:06:44.491938 22610 ts_tablet_manager.cc:1939] T c993920252fb442793b5bfd712ce6b85 P 6bde50f595cc4e798618343fd17526cc: Deleting consensus metadata
I20260809 02:06:44.492993 22612 ts_tablet_manager.cc:1918] T c993920252fb442793b5bfd712ce6b85 P 3d563744e8214676a11ac118a9f2dd27: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:06:44.493111 22612 log.cc:1196] T c993920252fb442793b5bfd712ce6b85 P 3d563744e8214676a11ac118a9f2dd27: Deleting WAL directory at /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786240941968412-1795-0/minicluster-data/ts-1/wal/wals/c993920252fb442793b5bfd712ce6b85
I20260809 02:06:44.493438 22612 ts_tablet_manager.cc:1939] T c993920252fb442793b5bfd712ce6b85 P 3d563744e8214676a11ac118a9f2dd27: Deleting consensus metadata
I20260809 02:06:44.493860 22080 catalog_manager.cc:4841] TS 6bde50f595cc4e798618343fd17526cc (127.1.192.195:34343): tablet c993920252fb442793b5bfd712ce6b85 (table test-table [id=13cbabc485af439e8c7c925bcbf3fe0c]) successfully deleted
I20260809 02:06:44.494724 22079 catalog_manager.cc:4841] TS 3d563744e8214676a11ac118a9f2dd27 (127.1.192.194:46369): tablet c993920252fb442793b5bfd712ce6b85 (table test-table [id=13cbabc485af439e8c7c925bcbf3fe0c]) successfully deleted
W20260809 02:06:44.500641 22080 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.192.193:41733 (error 108)
I20260809 02:06:44.500730 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 22299
I20260809 02:06:44.503541 22580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:44.503111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.193:41733 (local address 127.0.0.1:54286)
0809 02:06:44.503339 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0809 02:06:44.503423 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.193:41733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115}
W20260809 02:06:44.503733 22080 catalog_manager.cc:4584] TS 68fc9737ae65432db2124ea0ba869530 (127.1.192.193:41733): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c993920252fb442793b5bfd712ce6b85: Network error: Client connection negotiation failed: client connection to 127.1.192.193:41733: connect: Connection refused (error 111)
I20260809 02:06:44.507717 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 22435
I20260809 02:06:44.514460 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 22061
2026-08-09T02:06:44Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1370 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786240941968412-1795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:06:44 dist-test-slave-ss4n krb5kdc[22616](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:06:44 dist-test-slave-ss4n krb5kdc[22616](info): set up 2 sockets
Aug 09 02:06:44 dist-test-slave-ss4n krb5kdc[22616](info): commencing operation
krb5kdc: starting...
W20260809 02:06:45.673727 1795 mini_kdc.cc:121] Time spent starting KDC: real 1.136s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786240941968412-1795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786240941968412-1795-0/krb5kdc/test-user.keytab.
Aug 09 02:06:45 dist-test-slave-ss4n krb5kdc[22616](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241205, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:06:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:06:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.192.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.192.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.192.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab.
Entry for principal kudu/127.1.192.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab.
I20260809 02:06:45.818259 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786240941968412-1795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786240941968412-1795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786240941968412-1795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.192.254:36169
--webserver_interface=127.1.192.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab
--principal=kudu/127.1.192.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:44169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.192.254:36169
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:06:45.934983 22630 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:06:45.935307 22630 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:06:45.935387 22630 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:06:45.939700 22630 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:06:45.939793 22630 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:06:45.939831 22630 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:06:45.939884 22630 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:06:45.939934 22630 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:06:45.945612 22630 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:44169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786240941968412-1795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786240941968412-1795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.192.254:36169
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab
--principal=kudu/127.1.192.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.192.254:36169
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.192.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22630
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786240941968412-1795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:06:45.946830 22630 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:06:45.947896 22630 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:06:45.956591 22635 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:06:45.956667 22636 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:06:45.956606 22638 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:06:45.957062 22630 server_base.cc:1034] running on GCE node
I20260809 02:06:45.957754 22630 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:06:45.958865 22630 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:06:45.960070 22630 hybrid_clock.cc:648] HybridClock initialized: now 1786241205960052 us; error 36 us; skew 500 ppm
Aug 09 02:06:45 dist-test-slave-ss4n krb5kdc[22616](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241205, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:06:45.963404 22630 init.cc:377] Logged in from keytab as kudu/127.1.192.254@KRBTEST.COM (short username kudu)
I20260809 02:06:45.964377 22630 webserver.cc:460] Webserver started at http://127.1.192.254:37049/ using document root <none> and password file <none>
I20260809 02:06:45.964941 22630 fs_manager.cc:359] Metadata directory not provided
I20260809 02:06:45.965008 22630 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:06:45.965245 22630 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:06:45.967439 22630 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786240941968412-1795-0/minicluster-data/master-0/data/instance:
uuid: "e4a88438f700477cb49f0d27424c66e1"
format_stamp: "Formatted at 2026-08-09 02:06:45 on dist-test-slave-ss4n"
server_key: "544f4a6b3fd37c1c4c792ee29cadc4ce"
server_key_iv: "3bcc0c984c3cbe2a2f7bcca10400647f"
server_key_version: "encryptionkey@0"
I20260809 02:06:45.968011 22630 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786240941968412-1795-0/minicluster-data/master-0/wal/instance:
uuid: "e4a88438f700477cb49f0d27424c66e1"
format_stamp: "Formatted at 2026-08-09 02:06:45 on dist-test-slave-ss4n"
server_key: "544f4a6b3fd37c1c4c792ee29cadc4ce"
server_key_iv: "3bcc0c984c3cbe2a2f7bcca10400647f"
server_key_version: "encryptionkey@0"
I20260809 02:06:45.971906 22630 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 02:06:45.974820 22645 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:06:45.976189 22630 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 02:06:45.976331 22630 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786240941968412-1795-0/minicluster-data/master-0/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786240941968412-1795-0/minicluster-data/master-0/wal
uuid: "e4a88438f700477cb49f0d27424c66e1"
format_stamp: "Formatted at 2026-08-09 02:06:45 on dist-test-slave-ss4n"
server_key: "544f4a6b3fd37c1c4c792ee29cadc4ce"
server_key_iv: "3bcc0c984c3cbe2a2f7bcca10400647f"
server_key_version: "encryptionkey@0"
I20260809 02:06:45.976462 22630 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786240941968412-1795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786240941968412-1795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786240941968412-1795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:06:45.992189 22630 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:06:45.995738 22630 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:06:45.995978 22630 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:06:46.003952 22630 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.254:36169
I20260809 02:06:46.003978 22702 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.254:36169 every 8 connection(s)
I20260809 02:06:46.004930 22630 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
I20260809 02:06:46.006166 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 22630
I20260809 02:06:46.006312 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786240941968412-1795-0/minicluster-data/master-0/wal/instance
I20260809 02:06:46.006608 1795 external_mini_cluster.cc:1428] Setting key 7e65604115f95636665304c8b687eee4
I20260809 02:06:46.008008 22703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:06:46.015348 22703 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e4a88438f700477cb49f0d27424c66e1: Bootstrap starting.
Aug 09 02:06:46 dist-test-slave-ss4n krb5kdc[22616](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241205, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.192.254@KRBTEST.COM
I20260809 02:06:46.017951 22703 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e4a88438f700477cb49f0d27424c66e1: Neither blocks nor log segments found. Creating new log.
I20260809 02:06:46.018699 22703 log.cc:824] T 00000000000000000000000000000000 P e4a88438f700477cb49f0d27424c66e1: Log is configured to *not* fsync() on all Append() calls
I20260809 02:06:46.020645 22703 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e4a88438f700477cb49f0d27424c66e1: No bootstrap required, opened a new log
I20260809 02:06:46.022152 22706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:46.008048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36064 (local address 127.1.192.254:36169)
0809 02:06:46.008537 (+ 489us) server_negotiation.cc:206] Beginning negotiation
0809 02:06:46.008556 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0809 02:06:46.008598 (+ 42us) server_negotiation.cc:407] Connection header received
0809 02:06:46.009422 (+ 824us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:06:46.009445 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:06:46.009918 (+ 473us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:06:46.010305 (+ 387us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:06:46.011031 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:46.012576 (+ 1545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:46.013831 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:46.014298 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:46.014427 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:46.017286 (+ 2859us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:06:46.017291 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:06:46.017304 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:06:46.017333 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:06:46.019286 (+ 1953us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:06:46.019737 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:06:46.019742 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:06:46.019746 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:06:46.019859 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:06:46.020181 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:06:46.020185 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:06:46.020186 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:06:46.020583 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:06:46.020786 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0809 02:06:46.021079 (+ 293us) server_negotiation.cc:299] Negotiation successful
0809 02:06:46.021378 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":332,"thread_start_us":145,"threads_started":1}
I20260809 02:06:46.024534 22703 raft_consensus.cc:348] T 00000000000000000000000000000000 P e4a88438f700477cb49f0d27424c66e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4a88438f700477cb49f0d27424c66e1" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 36169 } }
I20260809 02:06:46.024776 22703 raft_consensus.cc:374] T 00000000000000000000000000000000 P e4a88438f700477cb49f0d27424c66e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:06:46.024852 22703 raft_consensus.cc:729] T 00000000000000000000000000000000 P e4a88438f700477cb49f0d27424c66e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4a88438f700477cb49f0d27424c66e1, State: Initialized, Role: FOLLOWER
I20260809 02:06:46.025261 22703 consensus_queue.cc:260] T 00000000000000000000000000000000 P e4a88438f700477cb49f0d27424c66e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4a88438f700477cb49f0d27424c66e1" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 36169 } }
I20260809 02:06:46.025408 22703 raft_consensus.cc:388] T 00000000000000000000000000000000 P e4a88438f700477cb49f0d27424c66e1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:06:46.025512 22703 raft_consensus.cc:482] T 00000000000000000000000000000000 P e4a88438f700477cb49f0d27424c66e1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:06:46.025631 22703 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e4a88438f700477cb49f0d27424c66e1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:06:46.026568 22703 raft_consensus.cc:504] T 00000000000000000000000000000000 P e4a88438f700477cb49f0d27424c66e1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4a88438f700477cb49f0d27424c66e1" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 36169 } }
I20260809 02:06:46.026896 22703 leader_election.cc:302] T 00000000000000000000000000000000 P e4a88438f700477cb49f0d27424c66e1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e4a88438f700477cb49f0d27424c66e1; no voters:
I20260809 02:06:46.027228 22703 leader_election.cc:288] T 00000000000000000000000000000000 P e4a88438f700477cb49f0d27424c66e1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:06:46.027386 22708 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e4a88438f700477cb49f0d27424c66e1 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:06:46.027647 22708 raft_consensus.cc:686] T 00000000000000000000000000000000 P e4a88438f700477cb49f0d27424c66e1 [term 1 LEADER]: Becoming Leader. State: Replica: e4a88438f700477cb49f0d27424c66e1, State: Running, Role: LEADER
I20260809 02:06:46.027954 22708 consensus_queue.cc:237] T 00000000000000000000000000000000 P e4a88438f700477cb49f0d27424c66e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4a88438f700477cb49f0d27424c66e1" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 36169 } }
I20260809 02:06:46.028528 22703 sys_catalog.cc:564] T 00000000000000000000000000000000 P e4a88438f700477cb49f0d27424c66e1 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:06:46.029711 22709 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4a88438f700477cb49f0d27424c66e1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e4a88438f700477cb49f0d27424c66e1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4a88438f700477cb49f0d27424c66e1" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 36169 } } }
I20260809 02:06:46.029855 22709 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4a88438f700477cb49f0d27424c66e1 [sys.catalog]: This master's current role is: LEADER
I20260809 02:06:46.030171 22710 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4a88438f700477cb49f0d27424c66e1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e4a88438f700477cb49f0d27424c66e1. Latest consensus state: current_term: 1 leader_uuid: "e4a88438f700477cb49f0d27424c66e1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4a88438f700477cb49f0d27424c66e1" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 36169 } } }
I20260809 02:06:46.030261 22710 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4a88438f700477cb49f0d27424c66e1 [sys.catalog]: This master's current role is: LEADER
I20260809 02:06:46.030360 22717 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:06:46.033179 22717 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:06:46.038538 22717 catalog_manager.cc:1347] Generated new cluster ID: 3e7c1ab52c014a2c8d02e16adb01ec94
I20260809 02:06:46.038619 22717 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:06:46.063302 22717 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:06:46.064757 22717 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:06:46.073930 22717 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e4a88438f700477cb49f0d27424c66e1: Generated new TSK 0
I20260809 02:06:46.074878 22717 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.192.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.192.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.192.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.193.keytab.
Entry for principal kudu/127.1.192.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.193.keytab.
I20260809 02:06:46.108529 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786240941968412-1795-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786240941968412-1795-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786240941968412-1795-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786240941968412-1795-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.192.193:0
--local_ip_for_outbound_sockets=127.1.192.193
--webserver_interface=127.1.192.193
--webserver_port=0
--tserver_master_addrs=127.1.192.254:36169
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.193.keytab
--principal=kudu/127.1.192.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:44169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:06:46.222797 22729 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:06:46.223136 22729 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:06:46.223212 22729 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:06:46.227521 22729 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:06:46.227622 22729 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:06:46.227718 22729 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.192.193
I20260809 02:06:46.233428 22729 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:44169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786240941968412-1795-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786240941968412-1795-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.193.keytab
--principal=kudu/127.1.192.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.192.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786240941968412-1795-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.192.193
--webserver_port=0
--tserver_master_addrs=127.1.192.254:36169
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22729
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.192.193
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786240941968412-1795-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:06:46.234614 22729 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:06:46.235720 22729 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:06:46.245978 22734 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:06:46.245998 22735 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:06:46.245997 22737 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:06:46.246297 22729 server_base.cc:1034] running on GCE node
I20260809 02:06:46.246840 22729 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:06:46.247659 22729 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:06:46.248864 22729 hybrid_clock.cc:648] HybridClock initialized: now 1786241206248841 us; error 38 us; skew 500 ppm
Aug 09 02:06:46 dist-test-slave-ss4n krb5kdc[22616](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241206, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:06:46.252105 22729 init.cc:377] Logged in from keytab as kudu/127.1.192.193@KRBTEST.COM (short username kudu)
I20260809 02:06:46.253165 22729 webserver.cc:460] Webserver started at http://127.1.192.193:34963/ using document root <none> and password file <none>
I20260809 02:06:46.253861 22729 fs_manager.cc:359] Metadata directory not provided
I20260809 02:06:46.253958 22729 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:06:46.254204 22729 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:06:46.256524 22729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786240941968412-1795-0/minicluster-data/ts-0/data/instance:
uuid: "2732b66ab9054fcebe974babb311dda8"
format_stamp: "Formatted at 2026-08-09 02:06:46 on dist-test-slave-ss4n"
server_key: "55516fd85866bad3f32b1d4afee8c8eb"
server_key_iv: "983ec21bd6433363ab9ff2d5860cad98"
server_key_version: "encryptionkey@0"
I20260809 02:06:46.257190 22729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786240941968412-1795-0/minicluster-data/ts-0/wal/instance:
uuid: "2732b66ab9054fcebe974babb311dda8"
format_stamp: "Formatted at 2026-08-09 02:06:46 on dist-test-slave-ss4n"
server_key: "55516fd85866bad3f32b1d4afee8c8eb"
server_key_iv: "983ec21bd6433363ab9ff2d5860cad98"
server_key_version: "encryptionkey@0"
I20260809 02:06:46.261586 22729 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260809 02:06:46.264344 22744 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:06:46.265698 22729 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.002s
I20260809 02:06:46.265838 22729 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786240941968412-1795-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786240941968412-1795-0/minicluster-data/ts-0/wal
uuid: "2732b66ab9054fcebe974babb311dda8"
format_stamp: "Formatted at 2026-08-09 02:06:46 on dist-test-slave-ss4n"
server_key: "55516fd85866bad3f32b1d4afee8c8eb"
server_key_iv: "983ec21bd6433363ab9ff2d5860cad98"
server_key_version: "encryptionkey@0"
I20260809 02:06:46.265980 22729 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786240941968412-1795-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786240941968412-1795-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786240941968412-1795-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:06:46.285827 22729 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:06:46.289108 22729 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:06:46.289328 22729 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:06:46.289984 22729 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:06:46.290849 22729 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:06:46.290918 22729 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:06:46.290987 22729 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:06:46.291038 22729 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:06:46.300450 22729 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.193:35713
I20260809 02:06:46.300493 22860 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.193:35713 every 8 connection(s)
I20260809 02:06:46.301409 22729 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786240941968412-1795-0/minicluster-data/ts-0/data/info.pb
I20260809 02:06:46.306620 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 22729
I20260809 02:06:46.306754 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786240941968412-1795-0/minicluster-data/ts-0/wal/instance
I20260809 02:06:46.307168 1795 external_mini_cluster.cc:1428] Setting key 7f7b45f2724c90f9d9013760d4c2e2c1
Aug 09 02:06:46 dist-test-slave-ss4n krb5kdc[22616](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241206, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.193@KRBTEST.COM for kudu/127.1.192.254@KRBTEST.COM
I20260809 02:06:46.318423 22706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:46.303378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.193:46555 (local address 127.1.192.254:36169)
0809 02:06:46.303626 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0809 02:06:46.303631 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:06:46.304903 (+ 1272us) server_negotiation.cc:407] Connection header received
0809 02:06:46.305997 (+ 1094us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:06:46.306000 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:06:46.306050 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:06:46.306132 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:06:46.307818 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:46.308370 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:46.310451 (+ 2081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:46.310705 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:46.310793 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:46.314247 (+ 3454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:06:46.314254 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:06:46.314258 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:06:46.314294 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:06:46.315903 (+ 1609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:06:46.316585 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:06:46.316590 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:06:46.316592 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:06:46.316682 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:06:46.317110 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:06:46.317112 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:06:46.317113 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:06:46.317311 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:06:46.317393 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 02:06:46.318155 (+ 762us) server_negotiation.cc:299] Negotiation successful
0809 02:06:46.318280 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
WARNING: no policy specified for kudu/127.1.192.194@KRBTEST.COM; defaulting to no policy
I20260809 02:06:46.319482 22863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:46.303797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:36169 (local address 127.1.192.193:46555)
0809 02:06:46.304682 (+ 885us) negotiation.cc:107] Waiting for socket to connect
0809 02:06:46.304754 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0809 02:06:46.305774 (+ 1020us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:06:46.306399 (+ 625us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:06:46.306406 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:06:46.306949 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:06:46.307622 (+ 673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:06:46.307640 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:06:46.308495 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:06:46.308499 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:06:46.310292 (+ 1793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:06:46.310309 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:06:46.310880 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:06:46.310888 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:06:46.311149 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:46.311947 (+ 798us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:06:46.311965 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:06:46.314001 (+ 2036us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:06:46.316037 (+ 2036us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:06:46.316046 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:06:46.316070 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:06:46.316464 (+ 394us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:06:46.316781 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:06:46.316784 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:06:46.316787 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:06:46.316993 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:06:46.317420 (+ 427us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:06:46.317428 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:06:46.317834 (+ 406us) client_negotiation.cc:769] Sending connection context
0809 02:06:46.318076 (+ 242us) client_negotiation.cc:240] Negotiation successful
0809 02:06:46.318465 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":657,"thread_start_us":153,"threads_started":1}
I20260809 02:06:46.320876 22861 heartbeater.cc:347] Connected to a master server at 127.1.192.254:36169
I20260809 02:06:46.321187 22861 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.192.194@KRBTEST.COM" created.
I20260809 02:06:46.321844 22861 heartbeater.cc:511] Master 127.1.192.254:36169 requested a full tablet report, sending...
I20260809 02:06:46.323553 22658 ts_manager.cc:194] Registered new tserver with Master: 2732b66ab9054fcebe974babb311dda8 (127.1.192.193:35713)
I20260809 02:06:46.325322 22658 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.192.193@KRBTEST.COM'} at 127.1.192.193:46555
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.192.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.194.keytab.
Entry for principal kudu/127.1.192.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.194.keytab.
I20260809 02:06:46.341699 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786240941968412-1795-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786240941968412-1795-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786240941968412-1795-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786240941968412-1795-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.192.194:0
--local_ip_for_outbound_sockets=127.1.192.194
--webserver_interface=127.1.192.194
--webserver_port=0
--tserver_master_addrs=127.1.192.254:36169
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.194.keytab
--principal=kudu/127.1.192.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:44169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:06:46.455849 22866 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:06:46.456148 22866 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:06:46.456195 22866 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:06:46.460369 22866 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:06:46.460440 22866 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:06:46.460502 22866 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.192.194
I20260809 02:06:46.466213 22866 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:44169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786240941968412-1795-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786240941968412-1795-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.194.keytab
--principal=kudu/127.1.192.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.192.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786240941968412-1795-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.192.194
--webserver_port=0
--tserver_master_addrs=127.1.192.254:36169
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22866
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.192.194
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786240941968412-1795-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:06:46.467352 22866 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:06:46.468446 22866 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 02:06:46.478919 22866 server_base.cc:1034] running on GCE node
W20260809 02:06:46.478935 22871 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:06:46.478928 22874 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:06:46.479295 22872 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:06:46.479792 22866 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:06:46.480603 22866 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:06:46.481818 22866 hybrid_clock.cc:648] HybridClock initialized: now 1786241206481782 us; error 50 us; skew 500 ppm
Aug 09 02:06:46 dist-test-slave-ss4n krb5kdc[22616](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241206, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:06:46.484900 22866 init.cc:377] Logged in from keytab as kudu/127.1.192.194@KRBTEST.COM (short username kudu)
I20260809 02:06:46.485914 22866 webserver.cc:460] Webserver started at http://127.1.192.194:46347/ using document root <none> and password file <none>
I20260809 02:06:46.486486 22866 fs_manager.cc:359] Metadata directory not provided
I20260809 02:06:46.486577 22866 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:06:46.486811 22866 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:06:46.488916 22866 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786240941968412-1795-0/minicluster-data/ts-1/data/instance:
uuid: "d480793ce9c443b18ea1afd13bc1f8d5"
format_stamp: "Formatted at 2026-08-09 02:06:46 on dist-test-slave-ss4n"
server_key: "8f2ce1b9497781c2cff2de4c106efa53"
server_key_iv: "9ef4eec3ab0e04fc2e35d338f6cb0952"
server_key_version: "encryptionkey@0"
I20260809 02:06:46.489553 22866 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786240941968412-1795-0/minicluster-data/ts-1/wal/instance:
uuid: "d480793ce9c443b18ea1afd13bc1f8d5"
format_stamp: "Formatted at 2026-08-09 02:06:46 on dist-test-slave-ss4n"
server_key: "8f2ce1b9497781c2cff2de4c106efa53"
server_key_iv: "9ef4eec3ab0e04fc2e35d338f6cb0952"
server_key_version: "encryptionkey@0"
I20260809 02:06:46.493275 22866 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260809 02:06:46.495880 22881 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:06:46.497198 22866 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 02:06:46.497339 22866 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786240941968412-1795-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786240941968412-1795-0/minicluster-data/ts-1/wal
uuid: "d480793ce9c443b18ea1afd13bc1f8d5"
format_stamp: "Formatted at 2026-08-09 02:06:46 on dist-test-slave-ss4n"
server_key: "8f2ce1b9497781c2cff2de4c106efa53"
server_key_iv: "9ef4eec3ab0e04fc2e35d338f6cb0952"
server_key_version: "encryptionkey@0"
I20260809 02:06:46.497465 22866 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786240941968412-1795-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786240941968412-1795-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786240941968412-1795-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:06:46.512105 22866 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:06:46.516006 22866 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:06:46.516237 22866 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:06:46.516909 22866 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:06:46.517879 22866 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:06:46.517953 22866 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:06:46.518033 22866 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:06:46.518078 22866 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:06:46.528308 22866 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.194:46393
I20260809 02:06:46.528349 23002 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.194:46393 every 8 connection(s)
I20260809 02:06:46.529386 22866 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786240941968412-1795-0/minicluster-data/ts-1/data/info.pb
I20260809 02:06:46.529794 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 22866
I20260809 02:06:46.529953 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786240941968412-1795-0/minicluster-data/ts-1/wal/instance
I20260809 02:06:46.530339 1795 external_mini_cluster.cc:1428] Setting key a506cb93635dabe8e5d8f4663a44d079
Aug 09 02:06:46 dist-test-slave-ss4n krb5kdc[22616](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241206, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.194@KRBTEST.COM for kudu/127.1.192.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.192.195@KRBTEST.COM; defaulting to no policy
I20260809 02:06:46.544494 22706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:46.531493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.194:46143 (local address 127.1.192.254:36169)
0809 02:06:46.531698 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0809 02:06:46.531702 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:06:46.532762 (+ 1060us) server_negotiation.cc:407] Connection header received
0809 02:06:46.533781 (+ 1019us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:06:46.533787 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:06:46.533859 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:06:46.533956 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:06:46.535659 (+ 1703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:46.536181 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:46.537500 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:46.537732 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:46.537856 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:46.540835 (+ 2979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:06:46.540838 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:06:46.540840 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:06:46.540867 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:06:46.542397 (+ 1530us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:06:46.542932 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:06:46.542936 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:06:46.542938 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:06:46.543024 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:06:46.543358 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:06:46.543361 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:06:46.543363 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:06:46.543567 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:06:46.543675 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 02:06:46.544183 (+ 508us) server_negotiation.cc:299] Negotiation successful
0809 02:06:46.544313 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.192.195@KRBTEST.COM" created.
I20260809 02:06:46.545409 23006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:46.532074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:36169 (local address 127.1.192.194:46143)
0809 02:06:46.532619 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0809 02:06:46.532663 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 02:06:46.533453 (+ 790us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:06:46.534110 (+ 657us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:06:46.534117 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:06:46.534744 (+ 627us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:06:46.535479 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:06:46.535500 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:06:46.536314 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:06:46.536318 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:06:46.537346 (+ 1028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:06:46.537356 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:06:46.537900 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:06:46.537904 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:06:46.538094 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:46.538789 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:06:46.538806 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:06:46.540629 (+ 1823us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:06:46.542528 (+ 1899us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:06:46.542534 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:06:46.542552 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:06:46.542824 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:06:46.543116 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:06:46.543120 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:06:46.543122 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:06:46.543264 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:06:46.543674 (+ 410us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:06:46.543681 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:06:46.543937 (+ 256us) client_negotiation.cc:769] Sending connection context
0809 02:06:46.544138 (+ 201us) client_negotiation.cc:240] Negotiation successful
0809 02:06:46.544480 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":152,"threads_started":1}
I20260809 02:06:46.546761 23003 heartbeater.cc:347] Connected to a master server at 127.1.192.254:36169
I20260809 02:06:46.547056 23003 heartbeater.cc:464] Registering TS with master...
I20260809 02:06:46.547603 23003 heartbeater.cc:511] Master 127.1.192.254:36169 requested a full tablet report, sending...
I20260809 02:06:46.548745 22658 ts_manager.cc:194] Registered new tserver with Master: d480793ce9c443b18ea1afd13bc1f8d5 (127.1.192.194:46393)
I20260809 02:06:46.549403 22658 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.192.194@KRBTEST.COM'} at 127.1.192.194:46143
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.192.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.195.keytab.
Entry for principal kudu/127.1.192.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.195.keytab.
I20260809 02:06:46.564587 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786240941968412-1795-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786240941968412-1795-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786240941968412-1795-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786240941968412-1795-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.192.195:0
--local_ip_for_outbound_sockets=127.1.192.195
--webserver_interface=127.1.192.195
--webserver_port=0
--tserver_master_addrs=127.1.192.254:36169
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.195.keytab
--principal=kudu/127.1.192.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:44169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:06:46.683799 23009 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:06:46.684072 23009 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:06:46.684116 23009 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:06:46.688314 23009 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:06:46.688388 23009 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:06:46.688477 23009 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.192.195
I20260809 02:06:46.694090 23009 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:44169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786240941968412-1795-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786240941968412-1795-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.195.keytab
--principal=kudu/127.1.192.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.192.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786240941968412-1795-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.192.195
--webserver_port=0
--tserver_master_addrs=127.1.192.254:36169
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23009
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.192.195
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786240941968412-1795-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:06:46.695309 23009 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:06:46.696360 23009 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:06:46.706736 23018 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:06:46.706844 23009 server_base.cc:1034] running on GCE node
W20260809 02:06:46.706712 23016 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:06:46.706696 23015 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:06:46.707442 23009 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:06:46.708204 23009 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:06:46.709398 23009 hybrid_clock.cc:648] HybridClock initialized: now 1786241206709369 us; error 44 us; skew 500 ppm
Aug 09 02:06:46 dist-test-slave-ss4n krb5kdc[22616](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241206, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:06:46.712435 23009 init.cc:377] Logged in from keytab as kudu/127.1.192.195@KRBTEST.COM (short username kudu)
I20260809 02:06:46.713352 23009 webserver.cc:460] Webserver started at http://127.1.192.195:44619/ using document root <none> and password file <none>
I20260809 02:06:46.713950 23009 fs_manager.cc:359] Metadata directory not provided
I20260809 02:06:46.714013 23009 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:06:46.714207 23009 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:06:46.716351 23009 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786240941968412-1795-0/minicluster-data/ts-2/data/instance:
uuid: "1daa5ec81aa14df59f8530eeb8c2fbb8"
format_stamp: "Formatted at 2026-08-09 02:06:46 on dist-test-slave-ss4n"
server_key: "b7bfbdad7c039882a1547a25a0e94f22"
server_key_iv: "28144f74a7c917179f01bc74f13e8aef"
server_key_version: "encryptionkey@0"
I20260809 02:06:46.716934 23009 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786240941968412-1795-0/minicluster-data/ts-2/wal/instance:
uuid: "1daa5ec81aa14df59f8530eeb8c2fbb8"
format_stamp: "Formatted at 2026-08-09 02:06:46 on dist-test-slave-ss4n"
server_key: "b7bfbdad7c039882a1547a25a0e94f22"
server_key_iv: "28144f74a7c917179f01bc74f13e8aef"
server_key_version: "encryptionkey@0"
I20260809 02:06:46.720778 23009 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 02:06:46.723446 23026 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:06:46.724747 23009 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 02:06:46.724854 23009 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786240941968412-1795-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786240941968412-1795-0/minicluster-data/ts-2/wal
uuid: "1daa5ec81aa14df59f8530eeb8c2fbb8"
format_stamp: "Formatted at 2026-08-09 02:06:46 on dist-test-slave-ss4n"
server_key: "b7bfbdad7c039882a1547a25a0e94f22"
server_key_iv: "28144f74a7c917179f01bc74f13e8aef"
server_key_version: "encryptionkey@0"
I20260809 02:06:46.724993 23009 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786240941968412-1795-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786240941968412-1795-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786240941968412-1795-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:06:46.740335 23009 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:06:46.743846 23009 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:06:46.744053 23009 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:06:46.744699 23009 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:06:46.745652 23009 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:06:46.745702 23009 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:06:46.745744 23009 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:06:46.745777 23009 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:06:46.755119 23009 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.195:41667
I20260809 02:06:46.755182 23144 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.195:41667 every 8 connection(s)
I20260809 02:06:46.756053 23009 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786240941968412-1795-0/minicluster-data/ts-2/data/info.pb
I20260809 02:06:46.762668 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 23009
I20260809 02:06:46.762830 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786240941968412-1795-0/minicluster-data/ts-2/wal/instance
I20260809 02:06:46.763134 1795 external_mini_cluster.cc:1428] Setting key 9d9597875629b2a88b7e500f8ac36508
Aug 09 02:06:46 dist-test-slave-ss4n krb5kdc[22616](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241206, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.195@KRBTEST.COM for kudu/127.1.192.254@KRBTEST.COM
I20260809 02:06:46.770367 22706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:46.758057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.195:58921 (local address 127.1.192.254:36169)
0809 02:06:46.758231 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0809 02:06:46.758235 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:06:46.759090 (+ 855us) server_negotiation.cc:407] Connection header received
0809 02:06:46.760395 (+ 1305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:06:46.760398 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:06:46.760460 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:06:46.760572 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:06:46.761786 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:46.762341 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:46.763708 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:46.763978 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:46.764110 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:46.766827 (+ 2717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:06:46.766831 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:06:46.766833 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:06:46.766859 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:06:46.768361 (+ 1502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:06:46.768833 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:06:46.768836 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:06:46.768837 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:06:46.768912 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:06:46.769226 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:06:46.769228 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:06:46.769229 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:06:46.769402 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:06:46.769537 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0809 02:06:46.770140 (+ 603us) server_negotiation.cc:299] Negotiation successful
0809 02:06:46.770245 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260809 02:06:46.771291 23147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:46.758428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:36169 (local address 127.1.192.195:58921)
0809 02:06:46.758940 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0809 02:06:46.758983 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 02:06:46.760181 (+ 1198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:06:46.760737 (+ 556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:06:46.760743 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:06:46.761113 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:06:46.761640 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:06:46.761657 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:06:46.762477 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:06:46.762480 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:06:46.763568 (+ 1088us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:06:46.763582 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:06:46.764178 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:06:46.764182 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:06:46.764344 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:46.764975 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:06:46.764996 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:06:46.766670 (+ 1674us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:06:46.768471 (+ 1801us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:06:46.768476 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:06:46.768489 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:06:46.768745 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:06:46.768991 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:06:46.768993 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:06:46.768994 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:06:46.769147 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:06:46.769558 (+ 411us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:06:46.769568 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:06:46.769832 (+ 264us) client_negotiation.cc:769] Sending connection context
0809 02:06:46.770091 (+ 259us) client_negotiation.cc:240] Negotiation successful
0809 02:06:46.770362 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":124,"threads_started":1}
I20260809 02:06:46.772444 23145 heartbeater.cc:347] Connected to a master server at 127.1.192.254:36169
I20260809 02:06:46.772689 23145 heartbeater.cc:464] Registering TS with master...
I20260809 02:06:46.773331 23145 heartbeater.cc:511] Master 127.1.192.254:36169 requested a full tablet report, sending...
I20260809 02:06:46.775046 22658 ts_manager.cc:194] Registered new tserver with Master: 1daa5ec81aa14df59f8530eeb8c2fbb8 (127.1.192.195:41667)
I20260809 02:06:46.775921 22658 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.192.195@KRBTEST.COM'} at 127.1.192.195:58921
I20260809 02:06:46.777426 1795 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 02:06:46.788691 22706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:46.779489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36076 (local address 127.1.192.254:36169)
0809 02:06:46.779624 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0809 02:06:46.779627 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:06:46.779991 (+ 364us) server_negotiation.cc:407] Connection header received
0809 02:06:46.780176 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:06:46.780179 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:06:46.780231 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:06:46.780310 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:06:46.781165 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:46.781783 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:46.782949 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:46.783222 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:46.783347 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:46.785043 (+ 1696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:06:46.785047 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:06:46.785050 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:06:46.785085 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:06:46.786802 (+ 1717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:06:46.787276 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:06:46.787281 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:06:46.787283 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:06:46.787374 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:06:46.787683 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:06:46.787687 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:06:46.787689 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:06:46.787951 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:06:46.788109 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0809 02:06:46.788368 (+ 259us) server_negotiation.cc:299] Negotiation successful
0809 02:06:46.788500 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 02:06:46.792275 22665 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36076:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 02:06:46.795555 22665 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:06:46.811368 23161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:46.805896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53262 (local address 127.1.192.193:35713)
0809 02:06:46.806254 (+ 358us) server_negotiation.cc:206] Beginning negotiation
0809 02:06:46.806276 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0809 02:06:46.806528 (+ 252us) server_negotiation.cc:407] Connection header received
0809 02:06:46.806693 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:06:46.806697 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:06:46.806829 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:06:46.806977 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:06:46.807209 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:46.807959 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:46.809667 (+ 1708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:46.810454 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:46.810566 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:46.810707 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0809 02:06:46.811092 (+ 385us) server_negotiation.cc:299] Negotiation successful
0809 02:06:46.811201 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":265,"thread_start_us":77,"threads_started":1}
I20260809 02:06:46.811448 23159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:46.804690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.195:41667 (local address 127.0.0.1:54340)
0809 02:06:46.805544 (+ 854us) negotiation.cc:107] Waiting for socket to connect
0809 02:06:46.805560 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 02:06:46.805645 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:06:46.806330 (+ 685us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:06:46.806333 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:06:46.806347 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:06:46.806445 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:06:46.806454 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:06:46.808236 (+ 1782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:06:46.808239 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:06:46.809368 (+ 1129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:06:46.809378 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:06:46.810580 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:06:46.810584 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:06:46.810811 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:46.810916 (+ 105us) client_negotiation.cc:769] Sending connection context
0809 02:06:46.811038 (+ 122us) client_negotiation.cc:240] Negotiation successful
0809 02:06:46.811257 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":764,"mutex_wait_us":674,"thread_start_us":120,"threads_started":1}
I20260809 02:06:46.811448 23160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:46.805679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.193:35713 (local address 127.0.0.1:53262)
0809 02:06:46.806425 (+ 746us) negotiation.cc:107] Waiting for socket to connect
0809 02:06:46.806437 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:06:46.806561 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:06:46.806957 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:06:46.806960 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:06:46.806978 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:06:46.807092 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:06:46.807100 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:06:46.808105 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:06:46.808108 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:06:46.809532 (+ 1424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:06:46.809544 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:06:46.810580 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:06:46.810584 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:06:46.810813 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:46.810916 (+ 103us) client_negotiation.cc:769] Sending connection context
0809 02:06:46.811038 (+ 122us) client_negotiation.cc:240] Negotiation successful
0809 02:06:46.811257 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":673,"mutex_wait_us":172,"thread_start_us":89,"threads_started":1}
I20260809 02:06:46.812086 23157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:46.804953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54340 (local address 127.1.192.195:41667)
0809 02:06:46.805471 (+ 518us) server_negotiation.cc:206] Beginning negotiation
0809 02:06:46.805510 (+ 39us) server_negotiation.cc:399] Waiting for connection header
0809 02:06:46.805631 (+ 121us) server_negotiation.cc:407] Connection header received
0809 02:06:46.805806 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:06:46.805810 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:06:46.806011 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:06:46.806161 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:06:46.806569 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:46.807813 (+ 1244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:46.809524 (+ 1711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:46.810416 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:46.810547 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:46.810707 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0809 02:06:46.811616 (+ 909us) server_negotiation.cc:299] Negotiation successful
0809 02:06:46.811914 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":417,"thread_start_us":129,"threads_started":1}
I20260809 02:06:46.812978 23156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:46.804690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.194:46393 (local address 127.0.0.1:46994)
0809 02:06:46.805061 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0809 02:06:46.805122 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0809 02:06:46.805254 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:06:46.806200 (+ 946us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:06:46.806207 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:06:46.806234 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:06:46.806373 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:06:46.806380 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:06:46.807959 (+ 1579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:06:46.807963 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:06:46.809532 (+ 1569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:06:46.809544 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:06:46.812556 (+ 3012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:06:46.812567 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:06:46.812657 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:46.812675 (+ 18us) client_negotiation.cc:769] Sending connection context
0809 02:06:46.812764 (+ 89us) client_negotiation.cc:240] Negotiation successful
0809 02:06:46.812839 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"mutex_wait_us":80,"thread_start_us":83,"threads_started":1}
I20260809 02:06:46.813052 23158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:46.804970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46994 (local address 127.1.192.194:46393)
0809 02:06:46.805686 (+ 716us) server_negotiation.cc:206] Beginning negotiation
0809 02:06:46.805700 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 02:06:46.805726 (+ 26us) server_negotiation.cc:407] Connection header received
0809 02:06:46.805803 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:06:46.805808 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:06:46.806011 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:06:46.806533 (+ 522us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:06:46.806706 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:46.807810 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:46.810782 (+ 2972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:46.811737 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:46.812039 (+ 302us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:46.812342 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0809 02:06:46.812801 (+ 459us) server_negotiation.cc:299] Negotiation successful
0809 02:06:46.812899 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":611,"thread_start_us":138,"threads_started":1}
I20260809 02:06:46.815197 23075 tablet_service.cc:1467] Processing CreateTablet for tablet dc4ec9f20d3b4917bd23b243bac5346f (DEFAULT_TABLE table=test-table [id=ca73cf4d67764633902ca05f09f6d443]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:06:46.815197 22794 tablet_service.cc:1467] Processing CreateTablet for tablet dc4ec9f20d3b4917bd23b243bac5346f (DEFAULT_TABLE table=test-table [id=ca73cf4d67764633902ca05f09f6d443]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:06:46.815557 22933 tablet_service.cc:1467] Processing CreateTablet for tablet dc4ec9f20d3b4917bd23b243bac5346f (DEFAULT_TABLE table=test-table [id=ca73cf4d67764633902ca05f09f6d443]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:06:46.816347 23075 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dc4ec9f20d3b4917bd23b243bac5346f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:06:46.816342 22794 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dc4ec9f20d3b4917bd23b243bac5346f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:06:46.816565 22933 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dc4ec9f20d3b4917bd23b243bac5346f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:06:46.822808 23162 tablet_bootstrap.cc:492] T dc4ec9f20d3b4917bd23b243bac5346f P d480793ce9c443b18ea1afd13bc1f8d5: Bootstrap starting.
I20260809 02:06:46.824131 23163 tablet_bootstrap.cc:492] T dc4ec9f20d3b4917bd23b243bac5346f P 2732b66ab9054fcebe974babb311dda8: Bootstrap starting.
I20260809 02:06:46.824349 23164 tablet_bootstrap.cc:492] T dc4ec9f20d3b4917bd23b243bac5346f P 1daa5ec81aa14df59f8530eeb8c2fbb8: Bootstrap starting.
I20260809 02:06:46.825570 23162 tablet_bootstrap.cc:654] T dc4ec9f20d3b4917bd23b243bac5346f P d480793ce9c443b18ea1afd13bc1f8d5: Neither blocks nor log segments found. Creating new log.
I20260809 02:06:46.826293 23164 tablet_bootstrap.cc:654] T dc4ec9f20d3b4917bd23b243bac5346f P 1daa5ec81aa14df59f8530eeb8c2fbb8: Neither blocks nor log segments found. Creating new log.
I20260809 02:06:46.826293 23163 tablet_bootstrap.cc:654] T dc4ec9f20d3b4917bd23b243bac5346f P 2732b66ab9054fcebe974babb311dda8: Neither blocks nor log segments found. Creating new log.
I20260809 02:06:46.826318 23162 log.cc:824] T dc4ec9f20d3b4917bd23b243bac5346f P d480793ce9c443b18ea1afd13bc1f8d5: Log is configured to *not* fsync() on all Append() calls
I20260809 02:06:46.827203 23163 log.cc:824] T dc4ec9f20d3b4917bd23b243bac5346f P 2732b66ab9054fcebe974babb311dda8: Log is configured to *not* fsync() on all Append() calls
I20260809 02:06:46.827461 23164 log.cc:824] T dc4ec9f20d3b4917bd23b243bac5346f P 1daa5ec81aa14df59f8530eeb8c2fbb8: Log is configured to *not* fsync() on all Append() calls
I20260809 02:06:46.829388 23163 tablet_bootstrap.cc:492] T dc4ec9f20d3b4917bd23b243bac5346f P 2732b66ab9054fcebe974babb311dda8: No bootstrap required, opened a new log
I20260809 02:06:46.829387 23162 tablet_bootstrap.cc:492] T dc4ec9f20d3b4917bd23b243bac5346f P d480793ce9c443b18ea1afd13bc1f8d5: No bootstrap required, opened a new log
I20260809 02:06:46.829629 23162 ts_tablet_manager.cc:1397] T dc4ec9f20d3b4917bd23b243bac5346f P d480793ce9c443b18ea1afd13bc1f8d5: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.003s
I20260809 02:06:46.829629 23163 ts_tablet_manager.cc:1397] T dc4ec9f20d3b4917bd23b243bac5346f P 2732b66ab9054fcebe974babb311dda8: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260809 02:06:46.830704 23164 tablet_bootstrap.cc:492] T dc4ec9f20d3b4917bd23b243bac5346f P 1daa5ec81aa14df59f8530eeb8c2fbb8: No bootstrap required, opened a new log
I20260809 02:06:46.830921 23164 ts_tablet_manager.cc:1397] T dc4ec9f20d3b4917bd23b243bac5346f P 1daa5ec81aa14df59f8530eeb8c2fbb8: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260809 02:06:46.833835 23162 raft_consensus.cc:348] T dc4ec9f20d3b4917bd23b243bac5346f P d480793ce9c443b18ea1afd13bc1f8d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d480793ce9c443b18ea1afd13bc1f8d5" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 46393 } } peers { permanent_uuid: "1daa5ec81aa14df59f8530eeb8c2fbb8" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 41667 } } peers { permanent_uuid: "2732b66ab9054fcebe974babb311dda8" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 35713 } }
I20260809 02:06:46.834165 23162 raft_consensus.cc:374] T dc4ec9f20d3b4917bd23b243bac5346f P d480793ce9c443b18ea1afd13bc1f8d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:06:46.834244 23162 raft_consensus.cc:729] T dc4ec9f20d3b4917bd23b243bac5346f P d480793ce9c443b18ea1afd13bc1f8d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d480793ce9c443b18ea1afd13bc1f8d5, State: Initialized, Role: FOLLOWER
I20260809 02:06:46.834189 23163 raft_consensus.cc:348] T dc4ec9f20d3b4917bd23b243bac5346f P 2732b66ab9054fcebe974babb311dda8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d480793ce9c443b18ea1afd13bc1f8d5" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 46393 } } peers { permanent_uuid: "1daa5ec81aa14df59f8530eeb8c2fbb8" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 41667 } } peers { permanent_uuid: "2732b66ab9054fcebe974babb311dda8" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 35713 } }
I20260809 02:06:46.834451 23163 raft_consensus.cc:374] T dc4ec9f20d3b4917bd23b243bac5346f P 2732b66ab9054fcebe974babb311dda8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:06:46.834503 23163 raft_consensus.cc:729] T dc4ec9f20d3b4917bd23b243bac5346f P 2732b66ab9054fcebe974babb311dda8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2732b66ab9054fcebe974babb311dda8, State: Initialized, Role: FOLLOWER
I20260809 02:06:46.834012 23164 raft_consensus.cc:348] T dc4ec9f20d3b4917bd23b243bac5346f P 1daa5ec81aa14df59f8530eeb8c2fbb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d480793ce9c443b18ea1afd13bc1f8d5" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 46393 } } peers { permanent_uuid: "1daa5ec81aa14df59f8530eeb8c2fbb8" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 41667 } } peers { permanent_uuid: "2732b66ab9054fcebe974babb311dda8" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 35713 } }
I20260809 02:06:46.834664 23164 raft_consensus.cc:374] T dc4ec9f20d3b4917bd23b243bac5346f P 1daa5ec81aa14df59f8530eeb8c2fbb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:06:46.834728 23164 raft_consensus.cc:729] T dc4ec9f20d3b4917bd23b243bac5346f P 1daa5ec81aa14df59f8530eeb8c2fbb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1daa5ec81aa14df59f8530eeb8c2fbb8, State: Initialized, Role: FOLLOWER
I20260809 02:06:46.834782 23162 consensus_queue.cc:260] T dc4ec9f20d3b4917bd23b243bac5346f P d480793ce9c443b18ea1afd13bc1f8d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d480793ce9c443b18ea1afd13bc1f8d5" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 46393 } } peers { permanent_uuid: "1daa5ec81aa14df59f8530eeb8c2fbb8" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 41667 } } peers { permanent_uuid: "2732b66ab9054fcebe974babb311dda8" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 35713 } }
I20260809 02:06:46.834906 23163 consensus_queue.cc:260] T dc4ec9f20d3b4917bd23b243bac5346f P 2732b66ab9054fcebe974babb311dda8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d480793ce9c443b18ea1afd13bc1f8d5" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 46393 } } peers { permanent_uuid: "1daa5ec81aa14df59f8530eeb8c2fbb8" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 41667 } } peers { permanent_uuid: "2732b66ab9054fcebe974babb311dda8" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 35713 } }
I20260809 02:06:46.835211 23164 consensus_queue.cc:260] T dc4ec9f20d3b4917bd23b243bac5346f P 1daa5ec81aa14df59f8530eeb8c2fbb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d480793ce9c443b18ea1afd13bc1f8d5" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 46393 } } peers { permanent_uuid: "1daa5ec81aa14df59f8530eeb8c2fbb8" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 41667 } } peers { permanent_uuid: "2732b66ab9054fcebe974babb311dda8" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 35713 } }
I20260809 02:06:46.835634 23003 heartbeater.cc:503] Master 127.1.192.254:36169 was elected leader, sending a full tablet report...
I20260809 02:06:46.835893 23162 ts_tablet_manager.cc:1428] T dc4ec9f20d3b4917bd23b243bac5346f P d480793ce9c443b18ea1afd13bc1f8d5: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260809 02:06:46.835893 23163 ts_tablet_manager.cc:1428] T dc4ec9f20d3b4917bd23b243bac5346f P 2732b66ab9054fcebe974babb311dda8: Time spent starting tablet: real 0.006s user 0.002s sys 0.004s
I20260809 02:06:46.836213 22861 heartbeater.cc:503] Master 127.1.192.254:36169 was elected leader, sending a full tablet report...
I20260809 02:06:46.836251 23164 ts_tablet_manager.cc:1428] T dc4ec9f20d3b4917bd23b243bac5346f P 1daa5ec81aa14df59f8530eeb8c2fbb8: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260809 02:06:46.836582 23145 heartbeater.cc:503] Master 127.1.192.254:36169 was elected leader, sending a full tablet report...
I20260809 02:06:46.974746 23170 raft_consensus.cc:482] T dc4ec9f20d3b4917bd23b243bac5346f P 1daa5ec81aa14df59f8530eeb8c2fbb8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:06:46.975018 23170 raft_consensus.cc:504] T dc4ec9f20d3b4917bd23b243bac5346f P 1daa5ec81aa14df59f8530eeb8c2fbb8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d480793ce9c443b18ea1afd13bc1f8d5" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 46393 } } peers { permanent_uuid: "1daa5ec81aa14df59f8530eeb8c2fbb8" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 41667 } } peers { permanent_uuid: "2732b66ab9054fcebe974babb311dda8" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 35713 } }
I20260809 02:06:46.976285 23170 leader_election.cc:288] T dc4ec9f20d3b4917bd23b243bac5346f P 1daa5ec81aa14df59f8530eeb8c2fbb8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d480793ce9c443b18ea1afd13bc1f8d5 (127.1.192.194:46393), 2732b66ab9054fcebe974babb311dda8 (127.1.192.193:35713)
I20260809 02:06:46.981529 23158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:46.976760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.195:47391 (local address 127.1.192.194:46393)
0809 02:06:46.976897 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0809 02:06:46.976902 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:06:46.976916 (+ 14us) server_negotiation.cc:407] Connection header received
0809 02:06:46.976974 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:06:46.976977 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:06:46.977037 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:06:46.977137 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:06:46.977521 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:46.978336 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:46.980120 (+ 1784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:46.980760 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:46.980898 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:46.980947 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 02:06:46.981272 (+ 325us) server_negotiation.cc:299] Negotiation successful
0809 02:06:46.981350 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 02:06:46.981547 23147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:46.976526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.194:46393 (local address 127.1.192.195:47391)
0809 02:06:46.976644 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0809 02:06:46.976662 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 02:06:46.976765 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:06:46.977187 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:06:46.977193 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:06:46.977212 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:06:46.977343 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:06:46.977348 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:06:46.978477 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:06:46.978481 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:06:46.979956 (+ 1475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:06:46.979968 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:06:46.980903 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:06:46.980907 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:06:46.981132 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:46.981169 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 02:06:46.981243 (+ 74us) client_negotiation.cc:240] Negotiation successful
0809 02:06:46.981360 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32,"mutex_wait_us":157}
I20260809 02:06:46.981604 23161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:46.976787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.195:41855 (local address 127.1.192.193:35713)
0809 02:06:46.976951 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0809 02:06:46.976956 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:06:46.977098 (+ 142us) server_negotiation.cc:407] Connection header received
0809 02:06:46.977246 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:06:46.977249 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:06:46.977299 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:06:46.977408 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:06:46.977693 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:46.978416 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:46.980120 (+ 1704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:46.980772 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:46.980893 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:46.980947 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 02:06:46.981272 (+ 325us) server_negotiation.cc:299] Negotiation successful
0809 02:06:46.981350 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260809 02:06:46.981555 23171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:46.976593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.193:35713 (local address 127.1.192.195:41855)
0809 02:06:46.977001 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0809 02:06:46.977013 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:06:46.977132 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:06:46.977413 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:06:46.977417 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:06:46.977437 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:06:46.977592 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:06:46.977597 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:06:46.978499 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:06:46.978502 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:06:46.979957 (+ 1455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:06:46.979968 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:06:46.980903 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:06:46.980907 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:06:46.981132 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:46.981160 (+ 28us) client_negotiation.cc:769] Sending connection context
0809 02:06:46.981243 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 02:06:46.981375 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":351,"mutex_wait_us":200,"thread_start_us":124,"threads_started":1}
I20260809 02:06:46.982398 22955 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc4ec9f20d3b4917bd23b243bac5346f" candidate_uuid: "1daa5ec81aa14df59f8530eeb8c2fbb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d480793ce9c443b18ea1afd13bc1f8d5" is_pre_election: true
I20260809 02:06:46.982448 22815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc4ec9f20d3b4917bd23b243bac5346f" candidate_uuid: "1daa5ec81aa14df59f8530eeb8c2fbb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2732b66ab9054fcebe974babb311dda8" is_pre_election: true
I20260809 02:06:46.982733 22955 raft_consensus.cc:2454] T dc4ec9f20d3b4917bd23b243bac5346f P d480793ce9c443b18ea1afd13bc1f8d5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1daa5ec81aa14df59f8530eeb8c2fbb8 in term 0.
I20260809 02:06:46.982745 22815 raft_consensus.cc:2454] T dc4ec9f20d3b4917bd23b243bac5346f P 2732b66ab9054fcebe974babb311dda8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1daa5ec81aa14df59f8530eeb8c2fbb8 in term 0.
I20260809 02:06:46.983283 23028 leader_election.cc:302] T dc4ec9f20d3b4917bd23b243bac5346f P 1daa5ec81aa14df59f8530eeb8c2fbb8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 1daa5ec81aa14df59f8530eeb8c2fbb8, 2732b66ab9054fcebe974babb311dda8, d480793ce9c443b18ea1afd13bc1f8d5; no voters:
I20260809 02:06:46.983587 23170 raft_consensus.cc:2781] T dc4ec9f20d3b4917bd23b243bac5346f P 1daa5ec81aa14df59f8530eeb8c2fbb8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:06:46.983683 23170 raft_consensus.cc:482] T dc4ec9f20d3b4917bd23b243bac5346f P 1daa5ec81aa14df59f8530eeb8c2fbb8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:06:46.983736 23170 raft_consensus.cc:3037] T dc4ec9f20d3b4917bd23b243bac5346f P 1daa5ec81aa14df59f8530eeb8c2fbb8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:06:46.984786 23170 raft_consensus.cc:504] T dc4ec9f20d3b4917bd23b243bac5346f P 1daa5ec81aa14df59f8530eeb8c2fbb8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d480793ce9c443b18ea1afd13bc1f8d5" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 46393 } } peers { permanent_uuid: "1daa5ec81aa14df59f8530eeb8c2fbb8" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 41667 } } peers { permanent_uuid: "2732b66ab9054fcebe974babb311dda8" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 35713 } }
I20260809 02:06:46.985147 23170 leader_election.cc:288] T dc4ec9f20d3b4917bd23b243bac5346f P 1daa5ec81aa14df59f8530eeb8c2fbb8 [CANDIDATE]: Term 1 election: Requested vote from peers d480793ce9c443b18ea1afd13bc1f8d5 (127.1.192.194:46393), 2732b66ab9054fcebe974babb311dda8 (127.1.192.193:35713)
I20260809 02:06:46.985589 22955 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc4ec9f20d3b4917bd23b243bac5346f" candidate_uuid: "1daa5ec81aa14df59f8530eeb8c2fbb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d480793ce9c443b18ea1afd13bc1f8d5"
I20260809 02:06:46.985606 22815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc4ec9f20d3b4917bd23b243bac5346f" candidate_uuid: "1daa5ec81aa14df59f8530eeb8c2fbb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2732b66ab9054fcebe974babb311dda8"
I20260809 02:06:46.985728 22955 raft_consensus.cc:3037] T dc4ec9f20d3b4917bd23b243bac5346f P d480793ce9c443b18ea1afd13bc1f8d5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:06:46.985728 22815 raft_consensus.cc:3037] T dc4ec9f20d3b4917bd23b243bac5346f P 2732b66ab9054fcebe974babb311dda8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:06:46.987120 22955 raft_consensus.cc:2454] T dc4ec9f20d3b4917bd23b243bac5346f P d480793ce9c443b18ea1afd13bc1f8d5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1daa5ec81aa14df59f8530eeb8c2fbb8 in term 1.
I20260809 02:06:46.987120 22815 raft_consensus.cc:2454] T dc4ec9f20d3b4917bd23b243bac5346f P 2732b66ab9054fcebe974babb311dda8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1daa5ec81aa14df59f8530eeb8c2fbb8 in term 1.
I20260809 02:06:46.987480 23029 leader_election.cc:302] T dc4ec9f20d3b4917bd23b243bac5346f P 1daa5ec81aa14df59f8530eeb8c2fbb8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1daa5ec81aa14df59f8530eeb8c2fbb8, 2732b66ab9054fcebe974babb311dda8; no voters:
I20260809 02:06:46.987733 23170 raft_consensus.cc:2781] T dc4ec9f20d3b4917bd23b243bac5346f P 1daa5ec81aa14df59f8530eeb8c2fbb8 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:06:46.987957 23170 raft_consensus.cc:686] T dc4ec9f20d3b4917bd23b243bac5346f P 1daa5ec81aa14df59f8530eeb8c2fbb8 [term 1 LEADER]: Becoming Leader. State: Replica: 1daa5ec81aa14df59f8530eeb8c2fbb8, State: Running, Role: LEADER
I20260809 02:06:46.988320 23170 consensus_queue.cc:237] T dc4ec9f20d3b4917bd23b243bac5346f P 1daa5ec81aa14df59f8530eeb8c2fbb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d480793ce9c443b18ea1afd13bc1f8d5" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 46393 } } peers { permanent_uuid: "1daa5ec81aa14df59f8530eeb8c2fbb8" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 41667 } } peers { permanent_uuid: "2732b66ab9054fcebe974babb311dda8" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 35713 } }
I20260809 02:06:46.991519 22665 catalog_manager.cc:5495] T dc4ec9f20d3b4917bd23b243bac5346f P 1daa5ec81aa14df59f8530eeb8c2fbb8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1daa5ec81aa14df59f8530eeb8c2fbb8 (127.1.192.195). New cstate: current_term: 1 leader_uuid: "1daa5ec81aa14df59f8530eeb8c2fbb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d480793ce9c443b18ea1afd13bc1f8d5" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 46393 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1daa5ec81aa14df59f8530eeb8c2fbb8" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 41667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2732b66ab9054fcebe974babb311dda8" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 35713 } health_report { overall_health: UNKNOWN } } }
W20260809 02:06:47.007071 23146 tablet.cc:2369] T dc4ec9f20d3b4917bd23b243bac5346f P 1daa5ec81aa14df59f8530eeb8c2fbb8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 02:06:47.031011 23004 tablet.cc:2369] T dc4ec9f20d3b4917bd23b243bac5346f P d480793ce9c443b18ea1afd13bc1f8d5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 02:06:47.049139 23157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:47.045348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54350 (local address 127.1.192.195:41667)
0809 02:06:47.045593 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0809 02:06:47.045596 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:06:47.045611 (+ 15us) server_negotiation.cc:407] Connection header received
0809 02:06:47.045694 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:06:47.045696 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:06:47.045765 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:06:47.045880 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:06:47.046171 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:47.046782 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:47.047656 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:47.047893 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:47.047980 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:47.048270 (+ 290us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:06:47.048780 (+ 510us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:06:47.048882 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 02:06:47.048942 (+ 60us) server_negotiation.cc:299] Negotiation successful
0809 02:06:47.048996 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
W20260809 02:06:47.052817 22862 tablet.cc:2369] T dc4ec9f20d3b4917bd23b243bac5346f P 2732b66ab9054fcebe974babb311dda8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 02:06:47.055253 22815 raft_consensus.cc:1261] T dc4ec9f20d3b4917bd23b243bac5346f P 2732b66ab9054fcebe974babb311dda8 [term 1 FOLLOWER]: Refusing update from remote peer 1daa5ec81aa14df59f8530eeb8c2fbb8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:06:47.055251 22955 raft_consensus.cc:1261] T dc4ec9f20d3b4917bd23b243bac5346f P d480793ce9c443b18ea1afd13bc1f8d5 [term 1 FOLLOWER]: Refusing update from remote peer 1daa5ec81aa14df59f8530eeb8c2fbb8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:06:47.055992 23172 consensus_queue.cc:1035] T dc4ec9f20d3b4917bd23b243bac5346f P 1daa5ec81aa14df59f8530eeb8c2fbb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d480793ce9c443b18ea1afd13bc1f8d5" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 46393 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:06:47.056182 23170 consensus_queue.cc:1035] T dc4ec9f20d3b4917bd23b243bac5346f P 1daa5ec81aa14df59f8530eeb8c2fbb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2732b66ab9054fcebe974babb311dda8" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 35713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:06:47.064591 23176 mvcc.cc:205] Tried to move back new op lower bound from 7316443984090529792 to 7316443983826808832. Current Snapshot: MvccSnapshot[applied={T|T < 7316443984090529792}]
I20260809 02:06:47.064990 23177 mvcc.cc:205] Tried to move back new op lower bound from 7316443984090529792 to 7316443983826808832. Current Snapshot: MvccSnapshot[applied={T|T < 7316443984090529792}]
I20260809 02:06:47.073635 22665 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36076:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:06:47.073864 22665 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36076:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:06:47.076750 22665 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e4a88438f700477cb49f0d27424c66e1: Sending DeleteTablet for 3 replicas of tablet dc4ec9f20d3b4917bd23b243bac5346f
I20260809 02:06:47.077674 22933 tablet_service.cc:1514] Processing DeleteTablet for tablet dc4ec9f20d3b4917bd23b243bac5346f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:06:47 UTC) from {username='kudu', principal='kudu/127.1.192.254@KRBTEST.COM'} at 127.0.0.1:46994
I20260809 02:06:47.077697 23075 tablet_service.cc:1514] Processing DeleteTablet for tablet dc4ec9f20d3b4917bd23b243bac5346f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:06:47 UTC) from {username='kudu', principal='kudu/127.1.192.254@KRBTEST.COM'} at 127.0.0.1:54340
I20260809 02:06:47.078270 22794 tablet_service.cc:1514] Processing DeleteTablet for tablet dc4ec9f20d3b4917bd23b243bac5346f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:06:47 UTC) from {username='kudu', principal='kudu/127.1.192.254@KRBTEST.COM'} at 127.0.0.1:53262
I20260809 02:06:47.078804 23187 tablet_replica.cc:330] T dc4ec9f20d3b4917bd23b243bac5346f P 2732b66ab9054fcebe974babb311dda8: stopping tablet replica
I20260809 02:06:47.078926 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 22729
I20260809 02:06:47.086696 23186 tablet_replica.cc:330] T dc4ec9f20d3b4917bd23b243bac5346f P d480793ce9c443b18ea1afd13bc1f8d5: stopping tablet replica
I20260809 02:06:47.087028 23186 raft_consensus.cc:2229] T dc4ec9f20d3b4917bd23b243bac5346f P d480793ce9c443b18ea1afd13bc1f8d5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:06:47.087363 23186 raft_consensus.cc:2258] T dc4ec9f20d3b4917bd23b243bac5346f P d480793ce9c443b18ea1afd13bc1f8d5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:06:47.078805 23185 tablet_replica.cc:330] T dc4ec9f20d3b4917bd23b243bac5346f P 1daa5ec81aa14df59f8530eeb8c2fbb8: stopping tablet replica
I20260809 02:06:47.087857 23185 raft_consensus.cc:2229] T dc4ec9f20d3b4917bd23b243bac5346f P 1daa5ec81aa14df59f8530eeb8c2fbb8 [term 1 LEADER]: Raft consensus shutting down.
I20260809 02:06:47.088250 23185 raft_consensus.cc:2258] T dc4ec9f20d3b4917bd23b243bac5346f P 1daa5ec81aa14df59f8530eeb8c2fbb8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:06:47.089783 23185 ts_tablet_manager.cc:1905] T dc4ec9f20d3b4917bd23b243bac5346f P 1daa5ec81aa14df59f8530eeb8c2fbb8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:06:47.092034 23185 ts_tablet_manager.cc:1918] T dc4ec9f20d3b4917bd23b243bac5346f P 1daa5ec81aa14df59f8530eeb8c2fbb8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:06:47.092208 23185 log.cc:1196] T dc4ec9f20d3b4917bd23b243bac5346f P 1daa5ec81aa14df59f8530eeb8c2fbb8: Deleting WAL directory at /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786240941968412-1795-0/minicluster-data/ts-2/wal/wals/dc4ec9f20d3b4917bd23b243bac5346f
I20260809 02:06:47.092549 23185 ts_tablet_manager.cc:1939] T dc4ec9f20d3b4917bd23b243bac5346f P 1daa5ec81aa14df59f8530eeb8c2fbb8: Deleting consensus metadata
I20260809 02:06:47.093823 22649 catalog_manager.cc:4841] TS 1daa5ec81aa14df59f8530eeb8c2fbb8 (127.1.192.195:41667): tablet dc4ec9f20d3b4917bd23b243bac5346f (table test-table [id=ca73cf4d67764633902ca05f09f6d443]) successfully deleted
W20260809 02:06:47.094262 22649 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.192.193:35713 (error 108)
I20260809 02:06:47.094553 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 22866
I20260809 02:06:47.096387 23156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:47.096116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.193:35713 (local address 127.0.0.1:53264)
0809 02:06:47.096248 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0809 02:06:47.096314 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.193:35713: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260809 02:06:47.096529 22649 catalog_manager.cc:4584] TS 2732b66ab9054fcebe974babb311dda8 (127.1.192.193:35713): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet dc4ec9f20d3b4917bd23b243bac5346f: Network error: Client connection negotiation failed: client connection to 127.1.192.193:35713: connect: Connection refused (error 111)
I20260809 02:06:47.103049 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 23009
I20260809 02:06:47.103318 23156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:47.102971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.194:46393 (local address 127.0.0.1:47004)
0809 02:06:47.103129 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0809 02:06:47.103210 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.194:46393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260809 02:06:47.114084 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 22630
2026-08-09T02:06:47Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2602 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-09T02:06:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:06:47Z Disabled control of system clock
I20260809 02:06:47.149120 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786240941968412-1795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786240941968412-1795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786240941968412-1795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.192.254:34769
--webserver_interface=127.1.192.254
--webserver_port=0
--builtin_ntp_servers=127.1.192.212:41373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.192.254:34769
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 02:06:47.261168 23192 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:06:47.261448 23192 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:06:47.261510 23192 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:06:47.265712 23192 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:06:47.265779 23192 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:06:47.265795 23192 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:06:47.265812 23192 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:06:47.265827 23192 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:06:47.270753 23192 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:41373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786240941968412-1795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786240941968412-1795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.192.254:34769
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.192.254:34769
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.192.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23192
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786240941968412-1795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:06:47.271898 23192 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:06:47.273337 23192 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:06:47.282804 23200 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:06:47.282856 23197 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:06:47.282840 23198 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:06:47.283119 23192 server_base.cc:1034] running on GCE node
I20260809 02:06:47.283910 23192 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:06:47.285172 23192 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:06:47.286360 23192 hybrid_clock.cc:648] HybridClock initialized: now 1786241207286314 us; error 56 us; skew 500 ppm
I20260809 02:06:47.293404 23192 webserver.cc:460] Webserver started at http://127.1.192.254:35345/ using document root <none> and password file <none>
I20260809 02:06:47.294180 23192 fs_manager.cc:359] Metadata directory not provided
I20260809 02:06:47.294260 23192 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:06:47.294517 23192 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:06:47.296949 23192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786240941968412-1795-0/minicluster-data/master-0/data/instance:
uuid: "0cef3c16df2c4d659072bd4fe38ec216"
format_stamp: "Formatted at 2026-08-09 02:06:47 on dist-test-slave-ss4n"
server_key: "ca9180bc7847cc6fd8f92f945770c45b"
server_key_iv: "b2579254193b2640c25b3de7da43dc7c"
server_key_version: "encryptionkey@0"
I20260809 02:06:47.297672 23192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786240941968412-1795-0/minicluster-data/master-0/wal/instance:
uuid: "0cef3c16df2c4d659072bd4fe38ec216"
format_stamp: "Formatted at 2026-08-09 02:06:47 on dist-test-slave-ss4n"
server_key: "ca9180bc7847cc6fd8f92f945770c45b"
server_key_iv: "b2579254193b2640c25b3de7da43dc7c"
server_key_version: "encryptionkey@0"
I20260809 02:06:47.302037 23192 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 02:06:47.304767 23206 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:06:47.306301 23192 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 02:06:47.306434 23192 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786240941968412-1795-0/minicluster-data/master-0/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786240941968412-1795-0/minicluster-data/master-0/wal
uuid: "0cef3c16df2c4d659072bd4fe38ec216"
format_stamp: "Formatted at 2026-08-09 02:06:47 on dist-test-slave-ss4n"
server_key: "ca9180bc7847cc6fd8f92f945770c45b"
server_key_iv: "b2579254193b2640c25b3de7da43dc7c"
server_key_version: "encryptionkey@0"
I20260809 02:06:47.306610 23192 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786240941968412-1795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786240941968412-1795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786240941968412-1795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:06:47.330459 23192 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:06:47.331209 23192 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:06:47.331377 23192 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:06:47.338347 23192 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.254:34769
I20260809 02:06:47.338382 23263 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.254:34769 every 8 connection(s)
I20260809 02:06:47.339356 23192 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
I20260809 02:06:47.342806 23264 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:06:47.345846 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 23192
I20260809 02:06:47.346016 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786240941968412-1795-0/minicluster-data/master-0/wal/instance
I20260809 02:06:47.346382 1795 external_mini_cluster.cc:1428] Setting key e0bbaa96526de645f2d305be7d5aee71
I20260809 02:06:47.349627 23264 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0cef3c16df2c4d659072bd4fe38ec216: Bootstrap starting.
I20260809 02:06:47.352088 23264 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0cef3c16df2c4d659072bd4fe38ec216: Neither blocks nor log segments found. Creating new log.
I20260809 02:06:47.352917 23264 log.cc:824] T 00000000000000000000000000000000 P 0cef3c16df2c4d659072bd4fe38ec216: Log is configured to *not* fsync() on all Append() calls
I20260809 02:06:47.355319 23264 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0cef3c16df2c4d659072bd4fe38ec216: No bootstrap required, opened a new log
I20260809 02:06:47.357304 23267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:47.348067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35894 (local address 127.1.192.254:34769)
0809 02:06:47.348861 (+ 794us) server_negotiation.cc:206] Beginning negotiation
0809 02:06:47.348879 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0809 02:06:47.348922 (+ 43us) server_negotiation.cc:407] Connection header received
0809 02:06:47.349729 (+ 807us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:06:47.349749 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:06:47.350231 (+ 482us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:06:47.350615 (+ 384us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:06:47.351147 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:47.352327 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:47.353535 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:47.354089 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:47.354290 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:47.355350 (+ 1060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:06:47.355356 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:06:47.355369 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:06:47.355895 (+ 526us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:06:47.355978 (+ 83us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:06:47.356001 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:06:47.356118 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0809 02:06:47.356268 (+ 150us) server_negotiation.cc:299] Negotiation successful
0809 02:06:47.356552 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":416,"thread_start_us":142,"threads_started":1}
I20260809 02:06:47.359700 23264 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0cef3c16df2c4d659072bd4fe38ec216 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cef3c16df2c4d659072bd4fe38ec216" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 34769 } }
I20260809 02:06:47.359975 23264 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0cef3c16df2c4d659072bd4fe38ec216 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:06:47.360065 23264 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0cef3c16df2c4d659072bd4fe38ec216 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0cef3c16df2c4d659072bd4fe38ec216, State: Initialized, Role: FOLLOWER
I20260809 02:06:47.360512 23264 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0cef3c16df2c4d659072bd4fe38ec216 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cef3c16df2c4d659072bd4fe38ec216" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 34769 } }
I20260809 02:06:47.360675 23264 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0cef3c16df2c4d659072bd4fe38ec216 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:06:47.360764 23264 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0cef3c16df2c4d659072bd4fe38ec216 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:06:47.360888 23264 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0cef3c16df2c4d659072bd4fe38ec216 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:06:47.362095 23264 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0cef3c16df2c4d659072bd4fe38ec216 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cef3c16df2c4d659072bd4fe38ec216" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 34769 } }
I20260809 02:06:47.362452 23264 leader_election.cc:302] T 00000000000000000000000000000000 P 0cef3c16df2c4d659072bd4fe38ec216 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0cef3c16df2c4d659072bd4fe38ec216; no voters:
I20260809 02:06:47.362810 23264 leader_election.cc:288] T 00000000000000000000000000000000 P 0cef3c16df2c4d659072bd4fe38ec216 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:06:47.362915 23269 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0cef3c16df2c4d659072bd4fe38ec216 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:06:47.363194 23269 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0cef3c16df2c4d659072bd4fe38ec216 [term 1 LEADER]: Becoming Leader. State: Replica: 0cef3c16df2c4d659072bd4fe38ec216, State: Running, Role: LEADER
I20260809 02:06:47.363518 23269 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0cef3c16df2c4d659072bd4fe38ec216 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cef3c16df2c4d659072bd4fe38ec216" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 34769 } }
I20260809 02:06:47.364176 23264 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0cef3c16df2c4d659072bd4fe38ec216 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:06:47.365236 23270 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0cef3c16df2c4d659072bd4fe38ec216 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0cef3c16df2c4d659072bd4fe38ec216" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cef3c16df2c4d659072bd4fe38ec216" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 34769 } } }
I20260809 02:06:47.365397 23270 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0cef3c16df2c4d659072bd4fe38ec216 [sys.catalog]: This master's current role is: LEADER
I20260809 02:06:47.365769 23271 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0cef3c16df2c4d659072bd4fe38ec216 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0cef3c16df2c4d659072bd4fe38ec216. Latest consensus state: current_term: 1 leader_uuid: "0cef3c16df2c4d659072bd4fe38ec216" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cef3c16df2c4d659072bd4fe38ec216" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 34769 } } }
I20260809 02:06:47.365875 23271 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0cef3c16df2c4d659072bd4fe38ec216 [sys.catalog]: This master's current role is: LEADER
I20260809 02:06:47.366199 23278 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:06:47.369402 23278 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:06:47.375484 23278 catalog_manager.cc:1347] Generated new cluster ID: 6efda23f165b48edac71f39b1061966d
I20260809 02:06:47.375571 23278 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:06:47.395814 23278 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:06:47.397290 23278 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:06:47.418972 23278 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0cef3c16df2c4d659072bd4fe38ec216: Generated new TSK 0
I20260809 02:06:47.419903 23278 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:06:47.435048 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786240941968412-1795-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786240941968412-1795-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786240941968412-1795-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786240941968412-1795-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.192.193:0
--local_ip_for_outbound_sockets=127.1.192.193
--webserver_interface=127.1.192.193
--webserver_port=0
--tserver_master_addrs=127.1.192.254:34769
--builtin_ntp_servers=127.1.192.212:41373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 02:06:47.559288 23288 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:06:47.559573 23288 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:06:47.559614 23288 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:06:47.564020 23288 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:06:47.564090 23288 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:06:47.564183 23288 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.192.193
I20260809 02:06:47.569379 23288 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:41373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786240941968412-1795-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786240941968412-1795-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.192.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786240941968412-1795-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.192.193
--webserver_port=0
--tserver_master_addrs=127.1.192.254:34769
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23288
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.192.193
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786240941968412-1795-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:06:47.570598 23288 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:06:47.572003 23288 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:06:47.581910 23294 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:06:47.581930 23296 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:06:47.581930 23293 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:06:47.582585 23288 server_base.cc:1034] running on GCE node
I20260809 02:06:47.583070 23288 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:06:47.583778 23288 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:06:47.584976 23288 hybrid_clock.cc:648] HybridClock initialized: now 1786241207584949 us; error 47 us; skew 500 ppm
I20260809 02:06:47.586946 23288 webserver.cc:460] Webserver started at http://127.1.192.193:36699/ using document root <none> and password file <none>
I20260809 02:06:47.587652 23288 fs_manager.cc:359] Metadata directory not provided
I20260809 02:06:47.587745 23288 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:06:47.588016 23288 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:06:47.590204 23288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786240941968412-1795-0/minicluster-data/ts-0/data/instance:
uuid: "7f6407a5ebb84be9a9a760c47e5d6717"
format_stamp: "Formatted at 2026-08-09 02:06:47 on dist-test-slave-ss4n"
server_key: "d3c42bb63f605ecc9849db8801ab51eb"
server_key_iv: "bb5cd4c2cb4bb672d93bb4e3cbad6248"
server_key_version: "encryptionkey@0"
I20260809 02:06:47.590783 23288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786240941968412-1795-0/minicluster-data/ts-0/wal/instance:
uuid: "7f6407a5ebb84be9a9a760c47e5d6717"
format_stamp: "Formatted at 2026-08-09 02:06:47 on dist-test-slave-ss4n"
server_key: "d3c42bb63f605ecc9849db8801ab51eb"
server_key_iv: "bb5cd4c2cb4bb672d93bb4e3cbad6248"
server_key_version: "encryptionkey@0"
I20260809 02:06:47.594599 23288 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260809 02:06:47.597270 23303 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:06:47.598685 23288 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 02:06:47.598841 23288 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786240941968412-1795-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786240941968412-1795-0/minicluster-data/ts-0/wal
uuid: "7f6407a5ebb84be9a9a760c47e5d6717"
format_stamp: "Formatted at 2026-08-09 02:06:47 on dist-test-slave-ss4n"
server_key: "d3c42bb63f605ecc9849db8801ab51eb"
server_key_iv: "bb5cd4c2cb4bb672d93bb4e3cbad6248"
server_key_version: "encryptionkey@0"
I20260809 02:06:47.598973 23288 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786240941968412-1795-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786240941968412-1795-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786240941968412-1795-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:06:47.627333 23288 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:06:47.628149 23288 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:06:47.628376 23288 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:06:47.629067 23288 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:06:47.630236 23288 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:06:47.630313 23288 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:06:47.630393 23288 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:06:47.630437 23288 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:06:47.640604 23288 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.193:42067
I20260809 02:06:47.640628 23420 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.193:42067 every 8 connection(s)
I20260809 02:06:47.641713 23288 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786240941968412-1795-0/minicluster-data/ts-0/data/info.pb
I20260809 02:06:47.644307 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 23288
I20260809 02:06:47.644449 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786240941968412-1795-0/minicluster-data/ts-0/wal/instance
I20260809 02:06:47.644877 1795 external_mini_cluster.cc:1428] Setting key f9ee019c154a74e6b263f1a22b817bc1
I20260809 02:06:47.647363 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786240941968412-1795-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786240941968412-1795-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786240941968412-1795-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786240941968412-1795-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.192.194:0
--local_ip_for_outbound_sockets=127.1.192.194
--webserver_interface=127.1.192.194
--webserver_port=0
--tserver_master_addrs=127.1.192.254:34769
--builtin_ntp_servers=127.1.192.212:41373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 02:06:47.654052 23267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:47.643917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.193:58981 (local address 127.1.192.254:34769)
0809 02:06:47.644112 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 02:06:47.644117 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:06:47.645564 (+ 1447us) server_negotiation.cc:407] Connection header received
0809 02:06:47.647493 (+ 1929us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:06:47.647500 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:06:47.647586 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:06:47.647689 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:06:47.648671 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:47.649618 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:47.651526 (+ 1908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:47.651810 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:47.651916 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:47.653412 (+ 1496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:06:47.653416 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:06:47.653419 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:06:47.653434 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:06:47.653457 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:06:47.653465 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:06:47.653615 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0809 02:06:47.653851 (+ 236us) server_negotiation.cc:299] Negotiation successful
0809 02:06:47.653918 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 02:06:47.655413 23423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:47.644455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:34769 (local address 127.1.192.193:58981)
0809 02:06:47.645339 (+ 884us) negotiation.cc:107] Waiting for socket to connect
0809 02:06:47.645393 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 02:06:47.647239 (+ 1846us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:06:47.647879 (+ 640us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:06:47.647886 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:06:47.647974 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:06:47.648508 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:06:47.648522 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:06:47.649767 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:06:47.649773 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:06:47.651364 (+ 1591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:06:47.651380 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:06:47.652005 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:06:47.652014 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:06:47.652285 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:47.653026 (+ 741us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 02:06:47.653029 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:06:47.653101 (+ 72us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 02:06:47.653205 (+ 104us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:06:47.653625 (+ 420us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:06:47.653640 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:06:47.653649 (+ 9us) client_negotiation.cc:769] Sending connection context
0809 02:06:47.653812 (+ 163us) client_negotiation.cc:240] Negotiation successful
0809 02:06:47.654204 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":170,"threads_started":1}
I20260809 02:06:47.656884 23421 heartbeater.cc:347] Connected to a master server at 127.1.192.254:34769
I20260809 02:06:47.657184 23421 heartbeater.cc:464] Registering TS with master...
I20260809 02:06:47.657965 23421 heartbeater.cc:511] Master 127.1.192.254:34769 requested a full tablet report, sending...
I20260809 02:06:47.659808 23225 ts_manager.cc:194] Registered new tserver with Master: 7f6407a5ebb84be9a9a760c47e5d6717 (127.1.192.193:42067)
I20260809 02:06:47.661072 23225 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.192.193:58981
W20260809 02:06:47.770812 23424 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:06:47.771140 23424 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:06:47.771212 23424 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:06:47.775442 23424 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:06:47.775535 23424 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:06:47.775638 23424 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.192.194
I20260809 02:06:47.782032 23424 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:41373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786240941968412-1795-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786240941968412-1795-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.192.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786240941968412-1795-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.192.194
--webserver_port=0
--tserver_master_addrs=127.1.192.254:34769
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23424
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.192.194
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786240941968412-1795-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:06:47.783250 23424 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:06:47.784684 23424 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:06:47.794312 23432 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:06:47.794399 23430 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:06:47.794312 23429 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:06:47.794687 23424 server_base.cc:1034] running on GCE node
I20260809 02:06:47.795202 23424 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:06:47.795869 23424 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:06:47.797039 23424 hybrid_clock.cc:648] HybridClock initialized: now 1786241207797023 us; error 31 us; skew 500 ppm
I20260809 02:06:47.798954 23424 webserver.cc:460] Webserver started at http://127.1.192.194:43179/ using document root <none> and password file <none>
I20260809 02:06:47.799580 23424 fs_manager.cc:359] Metadata directory not provided
I20260809 02:06:47.799649 23424 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:06:47.799854 23424 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:06:47.802119 23424 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786240941968412-1795-0/minicluster-data/ts-1/data/instance:
uuid: "049f7d89be3149508b795934a4c0987d"
format_stamp: "Formatted at 2026-08-09 02:06:47 on dist-test-slave-ss4n"
server_key: "a5cf46e61374d099b7d00bfa83280a7a"
server_key_iv: "a0a839f3d3cdc7ffff51c430902e785a"
server_key_version: "encryptionkey@0"
I20260809 02:06:47.802744 23424 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786240941968412-1795-0/minicluster-data/ts-1/wal/instance:
uuid: "049f7d89be3149508b795934a4c0987d"
format_stamp: "Formatted at 2026-08-09 02:06:47 on dist-test-slave-ss4n"
server_key: "a5cf46e61374d099b7d00bfa83280a7a"
server_key_iv: "a0a839f3d3cdc7ffff51c430902e785a"
server_key_version: "encryptionkey@0"
I20260809 02:06:47.806702 23424 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260809 02:06:47.809430 23438 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:06:47.810793 23424 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 02:06:47.810921 23424 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786240941968412-1795-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786240941968412-1795-0/minicluster-data/ts-1/wal
uuid: "049f7d89be3149508b795934a4c0987d"
format_stamp: "Formatted at 2026-08-09 02:06:47 on dist-test-slave-ss4n"
server_key: "a5cf46e61374d099b7d00bfa83280a7a"
server_key_iv: "a0a839f3d3cdc7ffff51c430902e785a"
server_key_version: "encryptionkey@0"
I20260809 02:06:47.811036 23424 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786240941968412-1795-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786240941968412-1795-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786240941968412-1795-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:06:47.826645 23424 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:06:47.827361 23424 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:06:47.827560 23424 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:06:47.828172 23424 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:06:47.829119 23424 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:06:47.829175 23424 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:06:47.829218 23424 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:06:47.829270 23424 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:06:47.837895 23424 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.194:34863
I20260809 02:06:47.838009 23554 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.194:34863 every 8 connection(s)
I20260809 02:06:47.838855 23424 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786240941968412-1795-0/minicluster-data/ts-1/data/info.pb
I20260809 02:06:47.846145 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 23424
I20260809 02:06:47.846298 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786240941968412-1795-0/minicluster-data/ts-1/wal/instance
I20260809 02:06:47.846697 1795 external_mini_cluster.cc:1428] Setting key 8fe56ccc395efab39dfa21d0a9022050
I20260809 02:06:47.848687 23267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:47.840723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.194:58871 (local address 127.1.192.254:34769)
0809 02:06:47.840863 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0809 02:06:47.840867 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:06:47.842058 (+ 1191us) server_negotiation.cc:407] Connection header received
0809 02:06:47.843440 (+ 1382us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:06:47.843443 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:06:47.843499 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:06:47.843596 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:06:47.844491 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:47.845065 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:47.846305 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:47.846673 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:47.846814 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:47.848129 (+ 1315us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:06:47.848132 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:06:47.848134 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:06:47.848147 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:06:47.848170 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:06:47.848179 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:06:47.848284 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0809 02:06:47.848443 (+ 159us) server_negotiation.cc:299] Negotiation successful
0809 02:06:47.848508 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 02:06:47.849135 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786240941968412-1795-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786240941968412-1795-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786240941968412-1795-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786240941968412-1795-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.192.195:0
--local_ip_for_outbound_sockets=127.1.192.195
--webserver_interface=127.1.192.195
--webserver_port=0
--tserver_master_addrs=127.1.192.254:34769
--builtin_ntp_servers=127.1.192.212:41373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 02:06:47.849650 23557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:47.841173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:34769 (local address 127.1.192.194:58871)
0809 02:06:47.841909 (+ 736us) negotiation.cc:107] Waiting for socket to connect
0809 02:06:47.841951 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 02:06:47.843217 (+ 1266us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:06:47.843785 (+ 568us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:06:47.843792 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:06:47.843874 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:06:47.844350 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:06:47.844362 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:06:47.845178 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:06:47.845181 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:06:47.846139 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:06:47.846149 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:06:47.846865 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:06:47.846869 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:06:47.847011 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:47.847821 (+ 810us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 02:06:47.847823 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:06:47.847883 (+ 60us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 02:06:47.847961 (+ 78us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:06:47.848298 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:06:47.848312 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:06:47.848317 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 02:06:47.848442 (+ 125us) client_negotiation.cc:240] Negotiation successful
0809 02:06:47.848722 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":121,"threads_started":1}
I20260809 02:06:47.850831 23555 heartbeater.cc:347] Connected to a master server at 127.1.192.254:34769
I20260809 02:06:47.851058 23555 heartbeater.cc:464] Registering TS with master...
I20260809 02:06:47.851568 23555 heartbeater.cc:511] Master 127.1.192.254:34769 requested a full tablet report, sending...
I20260809 02:06:47.852545 23225 ts_manager.cc:194] Registered new tserver with Master: 049f7d89be3149508b795934a4c0987d (127.1.192.194:34863)
I20260809 02:06:47.853114 23225 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.192.194:58871
W20260809 02:06:47.969604 23559 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:06:47.969877 23559 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:06:47.969914 23559 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:06:47.973963 23559 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:06:47.974022 23559 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:06:47.974093 23559 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.192.195
I20260809 02:06:47.979000 23559 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:41373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786240941968412-1795-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786240941968412-1795-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.192.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786240941968412-1795-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.192.195
--webserver_port=0
--tserver_master_addrs=127.1.192.254:34769
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23559
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.192.195
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786240941968412-1795-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:06:47.980132 23559 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:06:47.981595 23559 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 02:06:47.991470 23559 server_base.cc:1034] running on GCE node
W20260809 02:06:47.991506 23564 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:06:47.991410 23565 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:06:47.991410 23567 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:06:47.992204 23559 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:06:47.992867 23559 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:06:47.994066 23559 hybrid_clock.cc:648] HybridClock initialized: now 1786241207994035 us; error 36 us; skew 500 ppm
I20260809 02:06:47.996081 23559 webserver.cc:460] Webserver started at http://127.1.192.195:42071/ using document root <none> and password file <none>
I20260809 02:06:47.996711 23559 fs_manager.cc:359] Metadata directory not provided
I20260809 02:06:47.996803 23559 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:06:47.997031 23559 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:06:47.999197 23559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786240941968412-1795-0/minicluster-data/ts-2/data/instance:
uuid: "de306195956642aba708a41dc22e96eb"
format_stamp: "Formatted at 2026-08-09 02:06:47 on dist-test-slave-ss4n"
server_key: "f7d0f04b3e815847736d5c812dc1e2bc"
server_key_iv: "217279ad9883a12936b10f0801d8667c"
server_key_version: "encryptionkey@0"
I20260809 02:06:47.999799 23559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786240941968412-1795-0/minicluster-data/ts-2/wal/instance:
uuid: "de306195956642aba708a41dc22e96eb"
format_stamp: "Formatted at 2026-08-09 02:06:47 on dist-test-slave-ss4n"
server_key: "f7d0f04b3e815847736d5c812dc1e2bc"
server_key_iv: "217279ad9883a12936b10f0801d8667c"
server_key_version: "encryptionkey@0"
I20260809 02:06:48.003851 23559 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 02:06:48.006374 23573 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:06:48.007664 23559 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 02:06:48.007795 23559 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786240941968412-1795-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786240941968412-1795-0/minicluster-data/ts-2/wal
uuid: "de306195956642aba708a41dc22e96eb"
format_stamp: "Formatted at 2026-08-09 02:06:47 on dist-test-slave-ss4n"
server_key: "f7d0f04b3e815847736d5c812dc1e2bc"
server_key_iv: "217279ad9883a12936b10f0801d8667c"
server_key_version: "encryptionkey@0"
I20260809 02:06:48.007910 23559 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786240941968412-1795-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786240941968412-1795-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786240941968412-1795-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:06:48.020856 23559 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:06:48.021626 23559 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:06:48.021834 23559 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:06:48.022469 23559 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:06:48.023447 23559 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:06:48.023528 23559 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:06:48.023593 23559 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:06:48.023643 23559 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:06:48.032970 23559 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.195:33035
I20260809 02:06:48.032997 23688 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.195:33035 every 8 connection(s)
I20260809 02:06:48.033949 23559 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786240941968412-1795-0/minicluster-data/ts-2/data/info.pb
I20260809 02:06:48.037412 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 23559
I20260809 02:06:48.037551 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786240941968412-1795-0/minicluster-data/ts-2/wal/instance
I20260809 02:06:48.037942 1795 external_mini_cluster.cc:1428] Setting key ddfada6114ab726d594776ab07ebc896
I20260809 02:06:48.043979 23267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:48.035895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.195:36383 (local address 127.1.192.254:34769)
0809 02:06:48.036094 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0809 02:06:48.036098 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:06:48.037123 (+ 1025us) server_negotiation.cc:407] Connection header received
0809 02:06:48.038923 (+ 1800us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:06:48.038927 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:06:48.039002 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:06:48.039092 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:06:48.040096 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:48.040743 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:48.041971 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:48.042215 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:48.042345 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:48.043467 (+ 1122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:06:48.043471 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:06:48.043474 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:06:48.043492 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:06:48.043511 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:06:48.043518 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:06:48.043645 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0809 02:06:48.043762 (+ 117us) server_negotiation.cc:299] Negotiation successful
0809 02:06:48.043830 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260809 02:06:48.044888 23691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:48.036314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:34769 (local address 127.1.192.195:36383)
0809 02:06:48.036958 (+ 644us) negotiation.cc:107] Waiting for socket to connect
0809 02:06:48.037002 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 02:06:48.038696 (+ 1694us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:06:48.039320 (+ 624us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:06:48.039330 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:06:48.039434 (+ 104us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:06:48.039952 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:06:48.039965 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:06:48.040878 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:06:48.040882 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:06:48.041848 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:06:48.041858 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:06:48.042425 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:06:48.042430 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:06:48.042596 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:48.043197 (+ 601us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 02:06:48.043198 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:06:48.043241 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 02:06:48.043310 (+ 69us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:06:48.043603 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:06:48.043612 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:06:48.043618 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 02:06:48.043764 (+ 146us) client_negotiation.cc:240] Negotiation successful
0809 02:06:48.044066 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":117,"threads_started":1}
I20260809 02:06:48.046034 23689 heartbeater.cc:347] Connected to a master server at 127.1.192.254:34769
I20260809 02:06:48.046257 23689 heartbeater.cc:464] Registering TS with master...
I20260809 02:06:48.046756 23689 heartbeater.cc:511] Master 127.1.192.254:34769 requested a full tablet report, sending...
I20260809 02:06:48.047755 23225 ts_manager.cc:194] Registered new tserver with Master: de306195956642aba708a41dc22e96eb (127.1.192.195:33035)
I20260809 02:06:48.048290 23225 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.192.195:36383
I20260809 02:06:48.052273 1795 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 02:06:48.058177 23267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:48.054338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35898 (local address 127.1.192.254:34769)
0809 02:06:48.054487 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0809 02:06:48.054493 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:06:48.054759 (+ 266us) server_negotiation.cc:407] Connection header received
0809 02:06:48.054859 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:06:48.054861 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:06:48.054906 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:06:48.054982 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:06:48.055227 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:48.055758 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:48.056627 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:48.056835 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:48.056924 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:48.057761 (+ 837us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:06:48.057763 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:06:48.057766 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:06:48.057778 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:06:48.057793 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:06:48.057800 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:06:48.057857 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 02:06:48.057952 (+ 95us) server_negotiation.cc:299] Negotiation successful
0809 02:06:48.058004 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 02:06:48.061122 23225 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35898:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 02:06:48.064262 23225 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:06:48.079629 23700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:48.072800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.195:33035 (local address 127.0.0.1:53918)
0809 02:06:48.073863 (+ 1063us) negotiation.cc:107] Waiting for socket to connect
0809 02:06:48.073881 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 02:06:48.073986 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:06:48.074910 (+ 924us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:06:48.074913 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:06:48.074942 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:06:48.076043 (+ 1101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:06:48.076050 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:06:48.077147 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:06:48.077150 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:06:48.078226 (+ 1076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:06:48.078237 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:06:48.079192 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:06:48.079196 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:06:48.079282 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:48.079306 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 02:06:48.079367 (+ 61us) client_negotiation.cc:240] Negotiation successful
0809 02:06:48.079433 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":643,"mutex_wait_us":1085,"spinlock_wait_cycles":7424,"thread_start_us":92,"threads_started":1}
I20260809 02:06:48.079821 23707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:48.073623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41862 (local address 127.1.192.193:42067)
0809 02:06:48.074073 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0809 02:06:48.074085 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 02:06:48.074337 (+ 252us) server_negotiation.cc:407] Connection header received
0809 02:06:48.074408 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:06:48.074412 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:06:48.074593 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:06:48.074753 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:06:48.075071 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:48.075888 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:48.078115 (+ 2227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:48.078777 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:48.078867 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:48.079011 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0809 02:06:48.079534 (+ 523us) server_negotiation.cc:299] Negotiation successful
0809 02:06:48.079641 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346,"thread_start_us":121,"threads_started":1}
I20260809 02:06:48.080133 23702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:48.072958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53918 (local address 127.1.192.195:33035)
0809 02:06:48.074374 (+ 1416us) server_negotiation.cc:206] Beginning negotiation
0809 02:06:48.074388 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 02:06:48.074415 (+ 27us) server_negotiation.cc:407] Connection header received
0809 02:06:48.074499 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:06:48.074506 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:06:48.074729 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:06:48.074888 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:06:48.076176 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:48.077024 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:48.078358 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:48.079058 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:48.079178 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:48.079376 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0809 02:06:48.079432 (+ 56us) server_negotiation.cc:299] Negotiation successful
0809 02:06:48.079540 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1303,"thread_start_us":157,"threads_started":1}
I20260809 02:06:48.080468 23701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:48.073158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.193:42067 (local address 127.0.0.1:41862)
0809 02:06:48.074128 (+ 970us) negotiation.cc:107] Waiting for socket to connect
0809 02:06:48.074154 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0809 02:06:48.074243 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:06:48.074762 (+ 519us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:06:48.074769 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:06:48.074799 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:06:48.074954 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:06:48.074963 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:06:48.076020 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:06:48.076026 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:06:48.077979 (+ 1953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:06:48.077991 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:06:48.078919 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:06:48.078924 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:06:48.079062 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:48.079127 (+ 65us) client_negotiation.cc:769] Sending connection context
0809 02:06:48.079253 (+ 126us) client_negotiation.cc:240] Negotiation successful
0809 02:06:48.079388 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":907,"mutex_wait_us":195,"thread_start_us":99,"threads_started":1}
I20260809 02:06:48.081176 23705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:48.072815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.194:34863 (local address 127.0.0.1:57752)
0809 02:06:48.073723 (+ 908us) negotiation.cc:107] Waiting for socket to connect
0809 02:06:48.073779 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0809 02:06:48.073906 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:06:48.074909 (+ 1003us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:06:48.074913 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:06:48.074946 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:06:48.075084 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:06:48.075090 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:06:48.076208 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:06:48.076211 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:06:48.077983 (+ 1772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:06:48.077991 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:06:48.080781 (+ 2790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:06:48.080784 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:06:48.080875 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:48.080896 (+ 21us) client_negotiation.cc:769] Sending connection context
0809 02:06:48.080975 (+ 79us) client_negotiation.cc:240] Negotiation successful
0809 02:06:48.081039 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":793,"mutex_wait_us":175,"spinlock_wait_cycles":15872,"thread_start_us":113,"threads_started":1}
I20260809 02:06:48.081581 23706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:48.073551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57752 (local address 127.1.192.194:34863)
0809 02:06:48.074266 (+ 715us) server_negotiation.cc:206] Beginning negotiation
0809 02:06:48.074279 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 02:06:48.074305 (+ 26us) server_negotiation.cc:407] Connection header received
0809 02:06:48.074386 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:06:48.074391 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:06:48.074594 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:06:48.074753 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:06:48.075211 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:48.076063 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:48.079065 (+ 3002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:48.080403 (+ 1338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:48.080514 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:48.080712 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0809 02:06:48.081039 (+ 327us) server_negotiation.cc:299] Negotiation successful
0809 02:06:48.081151 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":615,"thread_start_us":159,"threads_started":1}
I20260809 02:06:48.083318 23621 tablet_service.cc:1467] Processing CreateTablet for tablet 96a51df769094060b0118504bf175a33 (DEFAULT_TABLE table=test-table [id=2c5d20d5f0e24fdbb2135ffd75eaa6fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:06:48.083460 23351 tablet_service.cc:1467] Processing CreateTablet for tablet 96a51df769094060b0118504bf175a33 (DEFAULT_TABLE table=test-table [id=2c5d20d5f0e24fdbb2135ffd75eaa6fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:06:48.084416 23621 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96a51df769094060b0118504bf175a33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:06:48.084597 23351 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96a51df769094060b0118504bf175a33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:06:48.084549 23487 tablet_service.cc:1467] Processing CreateTablet for tablet 96a51df769094060b0118504bf175a33 (DEFAULT_TABLE table=test-table [id=2c5d20d5f0e24fdbb2135ffd75eaa6fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:06:48.085703 23487 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96a51df769094060b0118504bf175a33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:06:48.091044 23708 tablet_bootstrap.cc:492] T 96a51df769094060b0118504bf175a33 P de306195956642aba708a41dc22e96eb: Bootstrap starting.
I20260809 02:06:48.092453 23709 tablet_bootstrap.cc:492] T 96a51df769094060b0118504bf175a33 P 7f6407a5ebb84be9a9a760c47e5d6717: Bootstrap starting.
I20260809 02:06:48.093108 23710 tablet_bootstrap.cc:492] T 96a51df769094060b0118504bf175a33 P 049f7d89be3149508b795934a4c0987d: Bootstrap starting.
I20260809 02:06:48.093694 23708 tablet_bootstrap.cc:654] T 96a51df769094060b0118504bf175a33 P de306195956642aba708a41dc22e96eb: Neither blocks nor log segments found. Creating new log.
I20260809 02:06:48.094662 23708 log.cc:824] T 96a51df769094060b0118504bf175a33 P de306195956642aba708a41dc22e96eb: Log is configured to *not* fsync() on all Append() calls
I20260809 02:06:48.095140 23709 tablet_bootstrap.cc:654] T 96a51df769094060b0118504bf175a33 P 7f6407a5ebb84be9a9a760c47e5d6717: Neither blocks nor log segments found. Creating new log.
I20260809 02:06:48.095278 23710 tablet_bootstrap.cc:654] T 96a51df769094060b0118504bf175a33 P 049f7d89be3149508b795934a4c0987d: Neither blocks nor log segments found. Creating new log.
I20260809 02:06:48.095963 23709 log.cc:824] T 96a51df769094060b0118504bf175a33 P 7f6407a5ebb84be9a9a760c47e5d6717: Log is configured to *not* fsync() on all Append() calls
I20260809 02:06:48.096144 23710 log.cc:824] T 96a51df769094060b0118504bf175a33 P 049f7d89be3149508b795934a4c0987d: Log is configured to *not* fsync() on all Append() calls
I20260809 02:06:48.097064 23708 tablet_bootstrap.cc:492] T 96a51df769094060b0118504bf175a33 P de306195956642aba708a41dc22e96eb: No bootstrap required, opened a new log
I20260809 02:06:48.097307 23708 ts_tablet_manager.cc:1397] T 96a51df769094060b0118504bf175a33 P de306195956642aba708a41dc22e96eb: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260809 02:06:48.098061 23709 tablet_bootstrap.cc:492] T 96a51df769094060b0118504bf175a33 P 7f6407a5ebb84be9a9a760c47e5d6717: No bootstrap required, opened a new log
I20260809 02:06:48.098248 23709 ts_tablet_manager.cc:1397] T 96a51df769094060b0118504bf175a33 P 7f6407a5ebb84be9a9a760c47e5d6717: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260809 02:06:48.101917 23708 raft_consensus.cc:348] T 96a51df769094060b0118504bf175a33 P de306195956642aba708a41dc22e96eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de306195956642aba708a41dc22e96eb" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 33035 } } peers { permanent_uuid: "049f7d89be3149508b795934a4c0987d" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 34863 } } peers { permanent_uuid: "7f6407a5ebb84be9a9a760c47e5d6717" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 42067 } }
I20260809 02:06:48.102265 23708 raft_consensus.cc:374] T 96a51df769094060b0118504bf175a33 P de306195956642aba708a41dc22e96eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:06:48.102339 23708 raft_consensus.cc:729] T 96a51df769094060b0118504bf175a33 P de306195956642aba708a41dc22e96eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de306195956642aba708a41dc22e96eb, State: Initialized, Role: FOLLOWER
I20260809 02:06:48.102653 23710 tablet_bootstrap.cc:492] T 96a51df769094060b0118504bf175a33 P 049f7d89be3149508b795934a4c0987d: No bootstrap required, opened a new log
I20260809 02:06:48.102869 23710 ts_tablet_manager.cc:1397] T 96a51df769094060b0118504bf175a33 P 049f7d89be3149508b795934a4c0987d: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.000s
I20260809 02:06:48.102811 23708 consensus_queue.cc:260] T 96a51df769094060b0118504bf175a33 P de306195956642aba708a41dc22e96eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de306195956642aba708a41dc22e96eb" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 33035 } } peers { permanent_uuid: "049f7d89be3149508b795934a4c0987d" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 34863 } } peers { permanent_uuid: "7f6407a5ebb84be9a9a760c47e5d6717" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 42067 } }
I20260809 02:06:48.102939 23709 raft_consensus.cc:348] T 96a51df769094060b0118504bf175a33 P 7f6407a5ebb84be9a9a760c47e5d6717 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de306195956642aba708a41dc22e96eb" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 33035 } } peers { permanent_uuid: "049f7d89be3149508b795934a4c0987d" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 34863 } } peers { permanent_uuid: "7f6407a5ebb84be9a9a760c47e5d6717" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 42067 } }
I20260809 02:06:48.103282 23709 raft_consensus.cc:374] T 96a51df769094060b0118504bf175a33 P 7f6407a5ebb84be9a9a760c47e5d6717 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:06:48.103338 23709 raft_consensus.cc:729] T 96a51df769094060b0118504bf175a33 P 7f6407a5ebb84be9a9a760c47e5d6717 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f6407a5ebb84be9a9a760c47e5d6717, State: Initialized, Role: FOLLOWER
I20260809 02:06:48.103757 23709 consensus_queue.cc:260] T 96a51df769094060b0118504bf175a33 P 7f6407a5ebb84be9a9a760c47e5d6717 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de306195956642aba708a41dc22e96eb" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 33035 } } peers { permanent_uuid: "049f7d89be3149508b795934a4c0987d" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 34863 } } peers { permanent_uuid: "7f6407a5ebb84be9a9a760c47e5d6717" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 42067 } }
I20260809 02:06:48.103885 23708 ts_tablet_manager.cc:1428] T 96a51df769094060b0118504bf175a33 P de306195956642aba708a41dc22e96eb: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260809 02:06:48.104256 23689 heartbeater.cc:503] Master 127.1.192.254:34769 was elected leader, sending a full tablet report...
I20260809 02:06:48.104655 23709 ts_tablet_manager.cc:1428] T 96a51df769094060b0118504bf175a33 P 7f6407a5ebb84be9a9a760c47e5d6717: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260809 02:06:48.105312 23421 heartbeater.cc:503] Master 127.1.192.254:34769 was elected leader, sending a full tablet report...
I20260809 02:06:48.106871 23710 raft_consensus.cc:348] T 96a51df769094060b0118504bf175a33 P 049f7d89be3149508b795934a4c0987d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de306195956642aba708a41dc22e96eb" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 33035 } } peers { permanent_uuid: "049f7d89be3149508b795934a4c0987d" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 34863 } } peers { permanent_uuid: "7f6407a5ebb84be9a9a760c47e5d6717" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 42067 } }
I20260809 02:06:48.107172 23710 raft_consensus.cc:374] T 96a51df769094060b0118504bf175a33 P 049f7d89be3149508b795934a4c0987d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:06:48.107254 23710 raft_consensus.cc:729] T 96a51df769094060b0118504bf175a33 P 049f7d89be3149508b795934a4c0987d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 049f7d89be3149508b795934a4c0987d, State: Initialized, Role: FOLLOWER
I20260809 02:06:48.107736 23710 consensus_queue.cc:260] T 96a51df769094060b0118504bf175a33 P 049f7d89be3149508b795934a4c0987d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de306195956642aba708a41dc22e96eb" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 33035 } } peers { permanent_uuid: "049f7d89be3149508b795934a4c0987d" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 34863 } } peers { permanent_uuid: "7f6407a5ebb84be9a9a760c47e5d6717" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 42067 } }
I20260809 02:06:48.108460 23555 heartbeater.cc:503] Master 127.1.192.254:34769 was elected leader, sending a full tablet report...
I20260809 02:06:48.108837 23710 ts_tablet_manager.cc:1428] T 96a51df769094060b0118504bf175a33 P 049f7d89be3149508b795934a4c0987d: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260809 02:06:48.125245 23714 raft_consensus.cc:482] T 96a51df769094060b0118504bf175a33 P de306195956642aba708a41dc22e96eb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:06:48.125455 23714 raft_consensus.cc:504] T 96a51df769094060b0118504bf175a33 P de306195956642aba708a41dc22e96eb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de306195956642aba708a41dc22e96eb" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 33035 } } peers { permanent_uuid: "049f7d89be3149508b795934a4c0987d" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 34863 } } peers { permanent_uuid: "7f6407a5ebb84be9a9a760c47e5d6717" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 42067 } }
I20260809 02:06:48.126646 23714 leader_election.cc:288] T 96a51df769094060b0118504bf175a33 P de306195956642aba708a41dc22e96eb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 049f7d89be3149508b795934a4c0987d (127.1.192.194:34863), 7f6407a5ebb84be9a9a760c47e5d6717 (127.1.192.193:42067)
I20260809 02:06:48.130743 23691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:48.126920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.193:42067 (local address 127.1.192.195:53133)
0809 02:06:48.127102 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0809 02:06:48.127118 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 02:06:48.127210 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:06:48.127518 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:06:48.127521 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:06:48.127539 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:06:48.127678 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:06:48.127686 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:06:48.128604 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:06:48.128607 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:06:48.129588 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:06:48.129597 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:06:48.130334 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:06:48.130338 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:06:48.130439 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:48.130465 (+ 26us) client_negotiation.cc:769] Sending connection context
0809 02:06:48.130540 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 02:06:48.130612 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"mutex_wait_us":3}
I20260809 02:06:48.130774 23707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:48.127009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.195:53133 (local address 127.1.192.193:42067)
0809 02:06:48.127132 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0809 02:06:48.127151 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0809 02:06:48.127167 (+ 16us) server_negotiation.cc:407] Connection header received
0809 02:06:48.127350 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:06:48.127354 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:06:48.127408 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:06:48.127496 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:06:48.127819 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:48.128497 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:48.129713 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:48.130224 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:48.130318 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:48.130357 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0809 02:06:48.130568 (+ 211us) server_negotiation.cc:299] Negotiation successful
0809 02:06:48.130641 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 02:06:48.131034 23706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:48.127286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.192.195:49491 (local address 127.1.192.194:34863)
0809 02:06:48.127411 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0809 02:06:48.127415 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:06:48.127625 (+ 210us) server_negotiation.cc:407] Connection header received
0809 02:06:48.127844 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:06:48.127847 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:06:48.127891 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:06:48.127947 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:06:48.128230 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:48.128834 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:48.129998 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:48.130452 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:48.130573 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:48.130616 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0809 02:06:48.130835 (+ 219us) server_negotiation.cc:299] Negotiation successful
0809 02:06:48.130924 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 02:06:48.131054 23717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:48.126926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.194:34863 (local address 127.1.192.195:49491)
0809 02:06:48.127482 (+ 556us) negotiation.cc:107] Waiting for socket to connect
0809 02:06:48.127496 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 02:06:48.127709 (+ 213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:06:48.127976 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:06:48.127978 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:06:48.127997 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:06:48.128124 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:06:48.128130 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:06:48.128937 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:06:48.128940 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:06:48.129873 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:06:48.129884 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:06:48.130606 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:06:48.130609 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:06:48.130698 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:48.130720 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 02:06:48.130799 (+ 79us) client_negotiation.cc:240] Negotiation successful
0809 02:06:48.130869 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480,"mutex_wait_us":122,"thread_start_us":136,"threads_started":1}
I20260809 02:06:48.131577 23373 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96a51df769094060b0118504bf175a33" candidate_uuid: "de306195956642aba708a41dc22e96eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f6407a5ebb84be9a9a760c47e5d6717" is_pre_election: true
I20260809 02:06:48.131656 23508 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96a51df769094060b0118504bf175a33" candidate_uuid: "de306195956642aba708a41dc22e96eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "049f7d89be3149508b795934a4c0987d" is_pre_election: true
I20260809 02:06:48.131896 23373 raft_consensus.cc:2454] T 96a51df769094060b0118504bf175a33 P 7f6407a5ebb84be9a9a760c47e5d6717 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de306195956642aba708a41dc22e96eb in term 0.
I20260809 02:06:48.131949 23508 raft_consensus.cc:2454] T 96a51df769094060b0118504bf175a33 P 049f7d89be3149508b795934a4c0987d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de306195956642aba708a41dc22e96eb in term 0.
I20260809 02:06:48.132416 23575 leader_election.cc:302] T 96a51df769094060b0118504bf175a33 P de306195956642aba708a41dc22e96eb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 049f7d89be3149508b795934a4c0987d, 7f6407a5ebb84be9a9a760c47e5d6717, de306195956642aba708a41dc22e96eb; no voters:
I20260809 02:06:48.132751 23714 raft_consensus.cc:2781] T 96a51df769094060b0118504bf175a33 P de306195956642aba708a41dc22e96eb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:06:48.132853 23714 raft_consensus.cc:482] T 96a51df769094060b0118504bf175a33 P de306195956642aba708a41dc22e96eb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:06:48.132915 23714 raft_consensus.cc:3037] T 96a51df769094060b0118504bf175a33 P de306195956642aba708a41dc22e96eb [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:06:48.134024 23714 raft_consensus.cc:504] T 96a51df769094060b0118504bf175a33 P de306195956642aba708a41dc22e96eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de306195956642aba708a41dc22e96eb" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 33035 } } peers { permanent_uuid: "049f7d89be3149508b795934a4c0987d" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 34863 } } peers { permanent_uuid: "7f6407a5ebb84be9a9a760c47e5d6717" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 42067 } }
I20260809 02:06:48.134369 23714 leader_election.cc:288] T 96a51df769094060b0118504bf175a33 P de306195956642aba708a41dc22e96eb [CANDIDATE]: Term 1 election: Requested vote from peers 049f7d89be3149508b795934a4c0987d (127.1.192.194:34863), 7f6407a5ebb84be9a9a760c47e5d6717 (127.1.192.193:42067)
I20260809 02:06:48.134727 23508 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96a51df769094060b0118504bf175a33" candidate_uuid: "de306195956642aba708a41dc22e96eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "049f7d89be3149508b795934a4c0987d"
I20260809 02:06:48.134755 23373 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96a51df769094060b0118504bf175a33" candidate_uuid: "de306195956642aba708a41dc22e96eb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f6407a5ebb84be9a9a760c47e5d6717"
I20260809 02:06:48.134836 23508 raft_consensus.cc:3037] T 96a51df769094060b0118504bf175a33 P 049f7d89be3149508b795934a4c0987d [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:06:48.134922 23373 raft_consensus.cc:3037] T 96a51df769094060b0118504bf175a33 P 7f6407a5ebb84be9a9a760c47e5d6717 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:06:48.135916 23508 raft_consensus.cc:2454] T 96a51df769094060b0118504bf175a33 P 049f7d89be3149508b795934a4c0987d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de306195956642aba708a41dc22e96eb in term 1.
I20260809 02:06:48.135962 23373 raft_consensus.cc:2454] T 96a51df769094060b0118504bf175a33 P 7f6407a5ebb84be9a9a760c47e5d6717 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de306195956642aba708a41dc22e96eb in term 1.
I20260809 02:06:48.136243 23575 leader_election.cc:302] T 96a51df769094060b0118504bf175a33 P de306195956642aba708a41dc22e96eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 049f7d89be3149508b795934a4c0987d, de306195956642aba708a41dc22e96eb; no voters:
I20260809 02:06:48.136420 23714 raft_consensus.cc:2781] T 96a51df769094060b0118504bf175a33 P de306195956642aba708a41dc22e96eb [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:06:48.136667 23714 raft_consensus.cc:686] T 96a51df769094060b0118504bf175a33 P de306195956642aba708a41dc22e96eb [term 1 LEADER]: Becoming Leader. State: Replica: de306195956642aba708a41dc22e96eb, State: Running, Role: LEADER
I20260809 02:06:48.136983 23714 consensus_queue.cc:237] T 96a51df769094060b0118504bf175a33 P de306195956642aba708a41dc22e96eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de306195956642aba708a41dc22e96eb" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 33035 } } peers { permanent_uuid: "049f7d89be3149508b795934a4c0987d" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 34863 } } peers { permanent_uuid: "7f6407a5ebb84be9a9a760c47e5d6717" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 42067 } }
I20260809 02:06:48.139915 23224 catalog_manager.cc:5495] T 96a51df769094060b0118504bf175a33 P de306195956642aba708a41dc22e96eb reported cstate change: term changed from 0 to 1, leader changed from <none> to de306195956642aba708a41dc22e96eb (127.1.192.195). New cstate: current_term: 1 leader_uuid: "de306195956642aba708a41dc22e96eb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de306195956642aba708a41dc22e96eb" member_type: VOTER last_known_addr { host: "127.1.192.195" port: 33035 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "049f7d89be3149508b795934a4c0987d" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 34863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7f6407a5ebb84be9a9a760c47e5d6717" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 42067 } health_report { overall_health: UNKNOWN } } }
W20260809 02:06:48.142951 23422 tablet.cc:2369] T 96a51df769094060b0118504bf175a33 P 7f6407a5ebb84be9a9a760c47e5d6717: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 02:06:48.164619 23702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:48.160800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53934 (local address 127.1.192.195:33035)
0809 02:06:48.160946 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0809 02:06:48.160951 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:06:48.160965 (+ 14us) server_negotiation.cc:407] Connection header received
0809 02:06:48.161196 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:06:48.161199 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:06:48.161254 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:06:48.161370 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:06:48.161667 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:48.162192 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:48.163062 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:48.163290 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:48.163418 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:48.163735 (+ 317us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:06:48.164213 (+ 478us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:06:48.164344 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0809 02:06:48.164391 (+ 47us) server_negotiation.cc:299] Negotiation successful
0809 02:06:48.164442 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 02:06:48.170969 23508 raft_consensus.cc:1261] T 96a51df769094060b0118504bf175a33 P 049f7d89be3149508b795934a4c0987d [term 1 FOLLOWER]: Refusing update from remote peer de306195956642aba708a41dc22e96eb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:06:48.170989 23373 raft_consensus.cc:1261] T 96a51df769094060b0118504bf175a33 P 7f6407a5ebb84be9a9a760c47e5d6717 [term 1 FOLLOWER]: Refusing update from remote peer de306195956642aba708a41dc22e96eb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:06:48.171805 23718 consensus_queue.cc:1035] T 96a51df769094060b0118504bf175a33 P de306195956642aba708a41dc22e96eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f6407a5ebb84be9a9a760c47e5d6717" member_type: VOTER last_known_addr { host: "127.1.192.193" port: 42067 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:06:48.172019 23714 consensus_queue.cc:1035] T 96a51df769094060b0118504bf175a33 P de306195956642aba708a41dc22e96eb [LEADER]: Connected to new peer: Peer: permanent_uuid: "049f7d89be3149508b795934a4c0987d" member_type: VOTER last_known_addr { host: "127.1.192.194" port: 34863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:06:48.180619 23722 mvcc.cc:205] Tried to move back new op lower bound from 7316443988659810304 to 7316443988531888128. Current Snapshot: MvccSnapshot[applied={T|T < 7316443988659810304}]
I20260809 02:06:48.182196 23723 mvcc.cc:205] Tried to move back new op lower bound from 7316443988659810304 to 7316443988531888128. Current Snapshot: MvccSnapshot[applied={T|T < 7316443988659810304}]
I20260809 02:06:48.189920 23225 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35898:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:06:48.190121 23225 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35898:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:06:48.192582 23225 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0cef3c16df2c4d659072bd4fe38ec216: Sending DeleteTablet for 3 replicas of tablet 96a51df769094060b0118504bf175a33
I20260809 02:06:48.193372 23621 tablet_service.cc:1514] Processing DeleteTablet for tablet 96a51df769094060b0118504bf175a33 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:06:48 UTC) from {username='slave'} at 127.0.0.1:53918
I20260809 02:06:48.193450 23351 tablet_service.cc:1514] Processing DeleteTablet for tablet 96a51df769094060b0118504bf175a33 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:06:48 UTC) from {username='slave'} at 127.0.0.1:41862
I20260809 02:06:48.193675 23487 tablet_service.cc:1514] Processing DeleteTablet for tablet 96a51df769094060b0118504bf175a33 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:06:48 UTC) from {username='slave'} at 127.0.0.1:57752
I20260809 02:06:48.193997 23733 tablet_replica.cc:330] T 96a51df769094060b0118504bf175a33 P 7f6407a5ebb84be9a9a760c47e5d6717: stopping tablet replica
I20260809 02:06:48.193995 23732 tablet_replica.cc:330] T 96a51df769094060b0118504bf175a33 P de306195956642aba708a41dc22e96eb: stopping tablet replica
I20260809 02:06:48.194118 23734 tablet_replica.cc:330] T 96a51df769094060b0118504bf175a33 P 049f7d89be3149508b795934a4c0987d: stopping tablet replica
I20260809 02:06:48.194324 23732 raft_consensus.cc:2229] T 96a51df769094060b0118504bf175a33 P de306195956642aba708a41dc22e96eb [term 1 LEADER]: Raft consensus shutting down.
I20260809 02:06:48.194332 23733 raft_consensus.cc:2229] T 96a51df769094060b0118504bf175a33 P 7f6407a5ebb84be9a9a760c47e5d6717 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:06:48.194437 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 23288
I20260809 02:06:48.194427 23734 raft_consensus.cc:2229] T 96a51df769094060b0118504bf175a33 P 049f7d89be3149508b795934a4c0987d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:06:48.195720 23734 raft_consensus.cc:2258] T 96a51df769094060b0118504bf175a33 P 049f7d89be3149508b795934a4c0987d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:06:48.195906 23732 raft_consensus.cc:2258] T 96a51df769094060b0118504bf175a33 P de306195956642aba708a41dc22e96eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:06:48.197662 23734 ts_tablet_manager.cc:1905] T 96a51df769094060b0118504bf175a33 P 049f7d89be3149508b795934a4c0987d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:06:48.198146 23732 ts_tablet_manager.cc:1905] T 96a51df769094060b0118504bf175a33 P de306195956642aba708a41dc22e96eb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:06:48.201318 23734 ts_tablet_manager.cc:1918] T 96a51df769094060b0118504bf175a33 P 049f7d89be3149508b795934a4c0987d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:06:48.201438 23734 log.cc:1196] T 96a51df769094060b0118504bf175a33 P 049f7d89be3149508b795934a4c0987d: Deleting WAL directory at /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786240941968412-1795-0/minicluster-data/ts-1/wal/wals/96a51df769094060b0118504bf175a33
I20260809 02:06:48.201823 23734 ts_tablet_manager.cc:1939] T 96a51df769094060b0118504bf175a33 P 049f7d89be3149508b795934a4c0987d: Deleting consensus metadata
I20260809 02:06:48.202809 23732 ts_tablet_manager.cc:1918] T 96a51df769094060b0118504bf175a33 P de306195956642aba708a41dc22e96eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:06:48.202980 23732 log.cc:1196] T 96a51df769094060b0118504bf175a33 P de306195956642aba708a41dc22e96eb: Deleting WAL directory at /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786240941968412-1795-0/minicluster-data/ts-2/wal/wals/96a51df769094060b0118504bf175a33
I20260809 02:06:48.203361 23732 ts_tablet_manager.cc:1939] T 96a51df769094060b0118504bf175a33 P de306195956642aba708a41dc22e96eb: Deleting consensus metadata
I20260809 02:06:48.203353 23208 catalog_manager.cc:4841] TS 049f7d89be3149508b795934a4c0987d (127.1.192.194:34863): tablet 96a51df769094060b0118504bf175a33 (table test-table [id=2c5d20d5f0e24fdbb2135ffd75eaa6fc]) successfully deleted
W20260809 02:06:48.204511 23209 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.192.193:42067 (error 108)
I20260809 02:06:48.204586 23210 catalog_manager.cc:4841] TS de306195956642aba708a41dc22e96eb (127.1.192.195:33035): tablet 96a51df769094060b0118504bf175a33 (table test-table [id=2c5d20d5f0e24fdbb2135ffd75eaa6fc]) successfully deleted
I20260809 02:06:48.205355 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 23424
I20260809 02:06:48.205687 23701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:48.205342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.193:42067 (local address 127.0.0.1:41870)
0809 02:06:48.205477 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0809 02:06:48.205582 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.193:42067: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260809 02:06:48.206791 23209 catalog_manager.cc:4584] TS 7f6407a5ebb84be9a9a760c47e5d6717 (127.1.192.193:42067): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 96a51df769094060b0118504bf175a33: Network error: Client connection negotiation failed: client connection to 127.1.192.193:42067: connect: Connection refused (error 111)
I20260809 02:06:48.215350 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 23559
I20260809 02:06:48.222781 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 23192
2026-08-09T02:06:48Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1105 ms)
[----------] 6 tests from EncryptionPolicyTest (11436 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786240941968412-1795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:06:48 dist-test-slave-ss4n krb5kdc[23739](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:06:48 dist-test-slave-ss4n krb5kdc[23739](info): set up 2 sockets
Aug 09 02:06:48 dist-test-slave-ss4n krb5kdc[23739](info): commencing operation
krb5kdc: starting...
W20260809 02:06:49.385051 1795 mini_kdc.cc:121] Time spent starting KDC: real 1.139s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786240941968412-1795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786240941968412-1795-0/krb5kdc/test-user.keytab.
Aug 09 02:06:49 dist-test-slave-ss4n krb5kdc[23739](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241209, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:06:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:06:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786240941968412-1795-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786240941968412-1795-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 02:06:49.529599 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786240941968412-1795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786240941968412-1795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786240941968412-1795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35143
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786240941968412-1795-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35143
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:06:49.648240 23755 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:06:49.648542 23755 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:06:49.648593 23755 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:06:49.652681 23755 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:06:49.652760 23755 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:06:49.652781 23755 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:06:49.652801 23755 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:06:49.652832 23755 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:06:49.658234 23755 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786240941968412-1795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786240941968412-1795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35143
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786240941968412-1795-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35143
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23755
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786240941968412-1795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:06:49.659263 23755 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:06:49.660246 23755 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:06:49.669422 23761 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:06:49.669457 23765 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:06:49.669425 23762 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:06:49.669902 23755 server_base.cc:1034] running on GCE node
I20260809 02:06:49.670503 23755 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:06:49.671734 23755 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:06:49.672919 23755 hybrid_clock.cc:648] HybridClock initialized: now 1786241209672883 us; error 52 us; skew 500 ppm
Aug 09 02:06:49 dist-test-slave-ss4n krb5kdc[23739](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241209, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:06:49.676209 23755 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 02:06:49.677301 23755 webserver.cc:460] Webserver started at http://127.0.0.1:42501/ using document root <none> and password file <none>
I20260809 02:06:49.678082 23755 fs_manager.cc:359] Metadata directory not provided
I20260809 02:06:49.678188 23755 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:06:49.678436 23755 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:06:49.680766 23755 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786240941968412-1795-0/minicluster-data/master-0/data/instance:
uuid: "5cb5c92b53514fa1893a133278d22c14"
format_stamp: "Formatted at 2026-08-09 02:06:49 on dist-test-slave-ss4n"
server_key: "e70dfc8a3c630d57801df5967bf24897"
server_key_iv: "e96ee735eaaaf2511d3631925ba028ec"
server_key_version: "encryptionkey@0"
I20260809 02:06:49.681421 23755 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786240941968412-1795-0/minicluster-data/master-0/wal/instance:
uuid: "5cb5c92b53514fa1893a133278d22c14"
format_stamp: "Formatted at 2026-08-09 02:06:49 on dist-test-slave-ss4n"
server_key: "e70dfc8a3c630d57801df5967bf24897"
server_key_iv: "e96ee735eaaaf2511d3631925ba028ec"
server_key_version: "encryptionkey@0"
I20260809 02:06:49.685701 23755 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260809 02:06:49.688429 23772 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:06:49.689843 23755 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 02:06:49.690008 23755 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786240941968412-1795-0/minicluster-data/master-0/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786240941968412-1795-0/minicluster-data/master-0/wal
uuid: "5cb5c92b53514fa1893a133278d22c14"
format_stamp: "Formatted at 2026-08-09 02:06:49 on dist-test-slave-ss4n"
server_key: "e70dfc8a3c630d57801df5967bf24897"
server_key_iv: "e96ee735eaaaf2511d3631925ba028ec"
server_key_version: "encryptionkey@0"
I20260809 02:06:49.690147 23755 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786240941968412-1795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786240941968412-1795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786240941968412-1795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:06:49.711688 23755 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:06:49.715195 23755 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:06:49.715444 23755 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:06:49.723913 23755 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35143
I20260809 02:06:49.723944 23829 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35143 every 8 connection(s)
I20260809 02:06:49.724979 23755 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
I20260809 02:06:49.727886 23830 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:06:49.728030 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 23755
I20260809 02:06:49.728191 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786240941968412-1795-0/minicluster-data/master-0/wal/instance
I20260809 02:06:49.728524 1795 external_mini_cluster.cc:1428] Setting key cd27d6a01649277daa37dfbc51d862bd
I20260809 02:06:49.734603 23830 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5cb5c92b53514fa1893a133278d22c14: Bootstrap starting.
I20260809 02:06:49.736886 23830 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5cb5c92b53514fa1893a133278d22c14: Neither blocks nor log segments found. Creating new log.
I20260809 02:06:49.737661 23830 log.cc:824] T 00000000000000000000000000000000 P 5cb5c92b53514fa1893a133278d22c14: Log is configured to *not* fsync() on all Append() calls
Aug 09 02:06:49 dist-test-slave-ss4n krb5kdc[23739](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241209, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 02:06:49.739641 23830 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5cb5c92b53514fa1893a133278d22c14: No bootstrap required, opened a new log
I20260809 02:06:49.743213 23830 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5cb5c92b53514fa1893a133278d22c14 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cb5c92b53514fa1893a133278d22c14" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35143 } }
I20260809 02:06:49.743477 23830 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5cb5c92b53514fa1893a133278d22c14 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:06:49.743562 23830 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5cb5c92b53514fa1893a133278d22c14 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cb5c92b53514fa1893a133278d22c14, State: Initialized, Role: FOLLOWER
I20260809 02:06:49.744051 23830 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5cb5c92b53514fa1893a133278d22c14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cb5c92b53514fa1893a133278d22c14" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35143 } }
I20260809 02:06:49.744204 23830 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5cb5c92b53514fa1893a133278d22c14 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:06:49.744288 23830 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5cb5c92b53514fa1893a133278d22c14 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:06:49.744405 23830 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5cb5c92b53514fa1893a133278d22c14 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:06:49.744792 23833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:49.730081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52903 (local address 127.0.0.1:35143)
0809 02:06:49.730566 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0809 02:06:49.730583 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 02:06:49.730623 (+ 40us) server_negotiation.cc:407] Connection header received
0809 02:06:49.731378 (+ 755us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:06:49.731395 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:06:49.731749 (+ 354us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:06:49.732118 (+ 369us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:06:49.733082 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:49.734233 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:49.735261 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:49.735777 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:49.735974 (+ 197us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:49.739012 (+ 3038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:06:49.739018 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:06:49.739032 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:06:49.739062 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:06:49.741069 (+ 2007us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:06:49.741760 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:06:49.741766 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:06:49.741771 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:06:49.741897 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:06:49.742325 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:06:49.742328 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:06:49.742330 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:06:49.742762 (+ 432us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:06:49.743027 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0809 02:06:49.743390 (+ 363us) server_negotiation.cc:299] Negotiation successful
0809 02:06:49.743674 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":141,"threads_started":1}
I20260809 02:06:49.745682 23830 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5cb5c92b53514fa1893a133278d22c14 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cb5c92b53514fa1893a133278d22c14" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35143 } }
I20260809 02:06:49.746035 23830 leader_election.cc:302] T 00000000000000000000000000000000 P 5cb5c92b53514fa1893a133278d22c14 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5cb5c92b53514fa1893a133278d22c14; no voters:
I20260809 02:06:49.746392 23830 leader_election.cc:288] T 00000000000000000000000000000000 P 5cb5c92b53514fa1893a133278d22c14 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:06:49.746512 23835 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5cb5c92b53514fa1893a133278d22c14 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:06:49.746707 23835 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5cb5c92b53514fa1893a133278d22c14 [term 1 LEADER]: Becoming Leader. State: Replica: 5cb5c92b53514fa1893a133278d22c14, State: Running, Role: LEADER
I20260809 02:06:49.747011 23835 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5cb5c92b53514fa1893a133278d22c14 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cb5c92b53514fa1893a133278d22c14" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35143 } }
I20260809 02:06:49.747752 23830 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5cb5c92b53514fa1893a133278d22c14 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:06:49.748219 23835 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5cb5c92b53514fa1893a133278d22c14 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5cb5c92b53514fa1893a133278d22c14. Latest consensus state: current_term: 1 leader_uuid: "5cb5c92b53514fa1893a133278d22c14" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cb5c92b53514fa1893a133278d22c14" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35143 } } }
I20260809 02:06:49.748340 23835 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5cb5c92b53514fa1893a133278d22c14 [sys.catalog]: This master's current role is: LEADER
I20260809 02:06:49.748409 1795 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 02:06:49.748871 23840 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:06:49.752297 23840 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:06:49.752313 23837 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5cb5c92b53514fa1893a133278d22c14 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5cb5c92b53514fa1893a133278d22c14" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cb5c92b53514fa1893a133278d22c14" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35143 } } }
I20260809 02:06:49.752481 23837 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5cb5c92b53514fa1893a133278d22c14 [sys.catalog]: This master's current role is: LEADER
I20260809 02:06:49.758920 23840 catalog_manager.cc:1347] Generated new cluster ID: 8a75513b3fca4e7fa9dd4e8a37dbfd10
I20260809 02:06:49.758991 23840 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:06:49.762576 23833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:49.751396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57671 (local address 127.0.0.1:35143)
0809 02:06:49.751524 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0809 02:06:49.751530 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:06:49.751545 (+ 15us) server_negotiation.cc:407] Connection header received
0809 02:06:49.751839 (+ 294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:06:49.751844 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:06:49.751918 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:06:49.751993 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:06:49.753785 (+ 1792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:49.754371 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:49.755639 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:49.756037 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:49.756122 (+ 85us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:49.758354 (+ 2232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:06:49.758359 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:06:49.758362 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:06:49.758395 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:06:49.760475 (+ 2080us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:06:49.761127 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:06:49.761131 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:06:49.761133 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:06:49.761222 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:06:49.761610 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:06:49.761614 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:06:49.761616 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:06:49.761825 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:06:49.761977 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0809 02:06:49.762303 (+ 326us) server_negotiation.cc:299] Negotiation successful
0809 02:06:49.762407 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 02:06:49.768399 23840 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:06:49.769629 23840 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:06:49.775372 23840 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5cb5c92b53514fa1893a133278d22c14: Generated new TSK 0
I20260809 02:06:49.776170 23840 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:06:49.783870 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 23755
W20260809 02:06:49.792793 23737 connection.cc:535] client connection to 127.0.0.1:35143 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35143): Connection reset by peer (error 104)
2026-08-09T02:06:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1561 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786240941968412-1795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:06:49 dist-test-slave-ss4n krb5kdc[23863](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:06:49 dist-test-slave-ss4n krb5kdc[23863](info): set up 2 sockets
Aug 09 02:06:49 dist-test-slave-ss4n krb5kdc[23863](info): commencing operation
krb5kdc: starting...
W20260809 02:06:50.912803 1795 mini_kdc.cc:121] Time spent starting KDC: real 1.105s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786240941968412-1795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786240941968412-1795-0/krb5kdc/test-user.keytab.
Aug 09 02:06:51 dist-test-slave-ss4n krb5kdc[23863](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241211, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:06:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:06:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786240941968412-1795-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786240941968412-1795-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 02:06:51.062382 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786240941968412-1795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786240941968412-1795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786240941968412-1795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33259
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786240941968412-1795-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33259
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:06:51.186913 23876 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:06:51.187232 23876 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:06:51.187278 23876 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:06:51.191470 23876 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:06:51.191545 23876 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:06:51.191565 23876 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:06:51.191581 23876 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:06:51.191596 23876 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:06:51.197258 23876 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786240941968412-1795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786240941968412-1795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33259
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786240941968412-1795-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33259
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23876
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786240941968412-1795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:06:51.198589 23876 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:06:51.199810 23876 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:06:51.210068 23884 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:06:51.210114 23881 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:06:51.210081 23882 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:06:51.210523 23876 server_base.cc:1034] running on GCE node
I20260809 02:06:51.211097 23876 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:06:51.212374 23876 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:06:51.213591 23876 hybrid_clock.cc:648] HybridClock initialized: now 1786241211213575 us; error 39 us; skew 500 ppm
Aug 09 02:06:51 dist-test-slave-ss4n krb5kdc[23863](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241211, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:06:51.216634 23876 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 02:06:51.217815 23876 webserver.cc:460] Webserver started at http://127.0.0.1:46709/ using document root <none> and password file <none>
I20260809 02:06:51.218464 23876 fs_manager.cc:359] Metadata directory not provided
I20260809 02:06:51.218535 23876 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:06:51.218767 23876 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:06:51.221035 23876 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786240941968412-1795-0/minicluster-data/master-0/data/instance:
uuid: "6d1fbd5dc851458c9b374616be5fadb2"
format_stamp: "Formatted at 2026-08-09 02:06:51 on dist-test-slave-ss4n"
server_key: "c2fd5651eb15e3f59e4bd4fde073d91d"
server_key_iv: "a1b24a07552f23fc0dfff15ac9cf5748"
server_key_version: "encryptionkey@0"
I20260809 02:06:51.221707 23876 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786240941968412-1795-0/minicluster-data/master-0/wal/instance:
uuid: "6d1fbd5dc851458c9b374616be5fadb2"
format_stamp: "Formatted at 2026-08-09 02:06:51 on dist-test-slave-ss4n"
server_key: "c2fd5651eb15e3f59e4bd4fde073d91d"
server_key_iv: "a1b24a07552f23fc0dfff15ac9cf5748"
server_key_version: "encryptionkey@0"
I20260809 02:06:51.225790 23876 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260809 02:06:51.228412 23891 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:06:51.229933 23876 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 02:06:51.230087 23876 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786240941968412-1795-0/minicluster-data/master-0/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786240941968412-1795-0/minicluster-data/master-0/wal
uuid: "6d1fbd5dc851458c9b374616be5fadb2"
format_stamp: "Formatted at 2026-08-09 02:06:51 on dist-test-slave-ss4n"
server_key: "c2fd5651eb15e3f59e4bd4fde073d91d"
server_key_iv: "a1b24a07552f23fc0dfff15ac9cf5748"
server_key_version: "encryptionkey@0"
I20260809 02:06:51.230238 23876 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786240941968412-1795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786240941968412-1795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786240941968412-1795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:06:51.252530 23876 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:06:51.255882 23876 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:06:51.256110 23876 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:06:51.264166 23876 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33259
I20260809 02:06:51.264186 23947 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33259 every 8 connection(s)
I20260809 02:06:51.265202 23876 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
I20260809 02:06:51.268299 23948 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:06:51.271591 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 23876
I20260809 02:06:51.271771 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786240941968412-1795-0/minicluster-data/master-0/wal/instance
I20260809 02:06:51.272127 1795 external_mini_cluster.cc:1428] Setting key e8d77c7bc13fc9dfb461fed7ca59f337
I20260809 02:06:51.275399 23948 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d1fbd5dc851458c9b374616be5fadb2: Bootstrap starting.
I20260809 02:06:51.278005 23948 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6d1fbd5dc851458c9b374616be5fadb2: Neither blocks nor log segments found. Creating new log.
I20260809 02:06:51.278894 23948 log.cc:824] T 00000000000000000000000000000000 P 6d1fbd5dc851458c9b374616be5fadb2: Log is configured to *not* fsync() on all Append() calls
I20260809 02:06:51.281383 23948 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d1fbd5dc851458c9b374616be5fadb2: No bootstrap required, opened a new log
I20260809 02:06:51.282234 23951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:51.273865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58005 (local address 127.0.0.1:33259)
0809 02:06:51.274427 (+ 562us) server_negotiation.cc:206] Beginning negotiation
0809 02:06:51.274444 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 02:06:51.274483 (+ 39us) server_negotiation.cc:407] Connection header received
0809 02:06:51.275193 (+ 710us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:06:51.275211 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:06:51.275608 (+ 397us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:06:51.275982 (+ 374us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:06:51.276420 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:51.277424 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:51.278462 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:51.278984 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:51.279174 (+ 190us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:51.280327 (+ 1153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:06:51.280332 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:06:51.280345 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:06:51.280776 (+ 431us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:06:51.280834 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:06:51.280859 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:06:51.280973 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0809 02:06:51.281219 (+ 246us) server_negotiation.cc:299] Negotiation successful
0809 02:06:51.281393 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":402,"thread_start_us":192,"threads_started":1}
I20260809 02:06:51.285871 1795 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 02:06:51.285750 23948 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6d1fbd5dc851458c9b374616be5fadb2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d1fbd5dc851458c9b374616be5fadb2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33259 } }
I20260809 02:06:51.286090 23948 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6d1fbd5dc851458c9b374616be5fadb2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:06:51.286163 23948 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6d1fbd5dc851458c9b374616be5fadb2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d1fbd5dc851458c9b374616be5fadb2, State: Initialized, Role: FOLLOWER
I20260809 02:06:51.286703 23948 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6d1fbd5dc851458c9b374616be5fadb2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d1fbd5dc851458c9b374616be5fadb2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33259 } }
I20260809 02:06:51.286886 23948 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6d1fbd5dc851458c9b374616be5fadb2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:06:51.286958 23948 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6d1fbd5dc851458c9b374616be5fadb2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:06:51.287066 23948 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6d1fbd5dc851458c9b374616be5fadb2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:06:51.288439 23948 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6d1fbd5dc851458c9b374616be5fadb2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d1fbd5dc851458c9b374616be5fadb2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33259 } }
I20260809 02:06:51.288895 23948 leader_election.cc:302] T 00000000000000000000000000000000 P 6d1fbd5dc851458c9b374616be5fadb2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6d1fbd5dc851458c9b374616be5fadb2; no voters:
I20260809 02:06:51.289273 23948 leader_election.cc:288] T 00000000000000000000000000000000 P 6d1fbd5dc851458c9b374616be5fadb2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:06:51.289377 23959 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6d1fbd5dc851458c9b374616be5fadb2 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:06:51.289677 23959 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6d1fbd5dc851458c9b374616be5fadb2 [term 1 LEADER]: Becoming Leader. State: Replica: 6d1fbd5dc851458c9b374616be5fadb2, State: Running, Role: LEADER
I20260809 02:06:51.290066 23959 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6d1fbd5dc851458c9b374616be5fadb2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d1fbd5dc851458c9b374616be5fadb2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33259 } }
I20260809 02:06:51.290725 23948 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6d1fbd5dc851458c9b374616be5fadb2 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:06:51.291339 23959 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d1fbd5dc851458c9b374616be5fadb2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6d1fbd5dc851458c9b374616be5fadb2. Latest consensus state: current_term: 1 leader_uuid: "6d1fbd5dc851458c9b374616be5fadb2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d1fbd5dc851458c9b374616be5fadb2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33259 } } }
I20260809 02:06:51.291499 23959 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d1fbd5dc851458c9b374616be5fadb2 [sys.catalog]: This master's current role is: LEADER
I20260809 02:06:51.291919 23966 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:06:51.291884 23960 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d1fbd5dc851458c9b374616be5fadb2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6d1fbd5dc851458c9b374616be5fadb2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d1fbd5dc851458c9b374616be5fadb2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33259 } } }
I20260809 02:06:51.292104 23960 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d1fbd5dc851458c9b374616be5fadb2 [sys.catalog]: This master's current role is: LEADER
I20260809 02:06:51.294389 23951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:51.288672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36731 (local address 127.0.0.1:33259)
0809 02:06:51.288819 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0809 02:06:51.288824 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:06:51.289253 (+ 429us) server_negotiation.cc:407] Connection header received
0809 02:06:51.289542 (+ 289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:06:51.289546 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:06:51.289608 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:06:51.289710 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:06:51.290207 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:51.290964 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:51.292194 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:51.292561 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:51.292680 (+ 119us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:51.293888 (+ 1208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:06:51.293891 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:06:51.293894 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:06:51.293910 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:06:51.293940 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:06:51.293955 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:06:51.294054 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 02:06:51.294198 (+ 144us) server_negotiation.cc:299] Negotiation successful
0809 02:06:51.294235 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 02:06:51.296051 23966 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:06:51.302564 23966 catalog_manager.cc:1347] Generated new cluster ID: b5cbb3feda504d26bb22fc4e558ce059
I20260809 02:06:51.302666 23966 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:06:51.318377 23966 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:06:51.319677 23966 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:06:51.327777 23966 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6d1fbd5dc851458c9b374616be5fadb2: Generated new TSK 0
I20260809 02:06:51.328727 23966 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:06:51.340395 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 23876
W20260809 02:06:51.348869 23861 connection.cc:535] client connection to 127.0.0.1:33259 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33259): Connection reset by peer (error 104)
2026-08-09T02:06:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1556 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786240941968412-1795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:06:51 dist-test-slave-ss4n krb5kdc[23982](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:06:51 dist-test-slave-ss4n krb5kdc[23982](info): set up 2 sockets
Aug 09 02:06:51 dist-test-slave-ss4n krb5kdc[23982](info): commencing operation
krb5kdc: starting...
W20260809 02:06:52.465583 1795 mini_kdc.cc:121] Time spent starting KDC: real 1.101s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786240941968412-1795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786240941968412-1795-0/krb5kdc/test-user.keytab.
Aug 09 02:06:52 dist-test-slave-ss4n krb5kdc[23982](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241212, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:06:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:06:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786240941968412-1795-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786240941968412-1795-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 02:06:52.614755 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786240941968412-1795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786240941968412-1795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786240941968412-1795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39651
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786240941968412-1795-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39255
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39651
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:06:52.731738 23995 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:06:52.732075 23995 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:06:52.732159 23995 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:06:52.736359 23995 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:06:52.736465 23995 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:06:52.736503 23995 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:06:52.736543 23995 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:06:52.736575 23995 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:06:52.742271 23995 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39255
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786240941968412-1795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786240941968412-1795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39651
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786240941968412-1795-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39651
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23995
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786240941968412-1795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:06:52.743520 23995 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:06:52.744647 23995 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 02:06:52.754092 23995 server_base.cc:1034] running on GCE node
W20260809 02:06:52.754055 24001 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:06:52.754046 24003 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:06:52.754021 24000 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:06:52.754917 23995 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:06:52.756110 23995 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:06:52.757287 23995 hybrid_clock.cc:648] HybridClock initialized: now 1786241212757270 us; error 34 us; skew 500 ppm
Aug 09 02:06:52 dist-test-slave-ss4n krb5kdc[23982](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241212, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:06:52.766238 23995 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 02:06:52.767400 23995 webserver.cc:460] Webserver started at http://127.0.0.1:40861/ using document root <none> and password file <none>
I20260809 02:06:52.768074 23995 fs_manager.cc:359] Metadata directory not provided
I20260809 02:06:52.768214 23995 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:06:52.768481 23995 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:06:52.770867 23995 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786240941968412-1795-0/minicluster-data/master-0/data/instance:
uuid: "c7a19f37c4474c34ba0540cbab499cc8"
format_stamp: "Formatted at 2026-08-09 02:06:52 on dist-test-slave-ss4n"
server_key: "dadfc72b33c5c5ddcadaa57ca4fa8032"
server_key_iv: "7711e5644c64776b0d74181b71cb9cfa"
server_key_version: "encryptionkey@0"
I20260809 02:06:52.771543 23995 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786240941968412-1795-0/minicluster-data/master-0/wal/instance:
uuid: "c7a19f37c4474c34ba0540cbab499cc8"
format_stamp: "Formatted at 2026-08-09 02:06:52 on dist-test-slave-ss4n"
server_key: "dadfc72b33c5c5ddcadaa57ca4fa8032"
server_key_iv: "7711e5644c64776b0d74181b71cb9cfa"
server_key_version: "encryptionkey@0"
I20260809 02:06:52.775844 23995 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 02:06:52.778853 24010 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:06:52.780331 23995 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 02:06:52.780596 23995 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786240941968412-1795-0/minicluster-data/master-0/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786240941968412-1795-0/minicluster-data/master-0/wal
uuid: "c7a19f37c4474c34ba0540cbab499cc8"
format_stamp: "Formatted at 2026-08-09 02:06:52 on dist-test-slave-ss4n"
server_key: "dadfc72b33c5c5ddcadaa57ca4fa8032"
server_key_iv: "7711e5644c64776b0d74181b71cb9cfa"
server_key_version: "encryptionkey@0"
I20260809 02:06:52.780766 23995 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786240941968412-1795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786240941968412-1795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786240941968412-1795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:06:52.786011 23995 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:06:52.789666 23995 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:06:52.789893 23995 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:06:52.797717 23995 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39651
I20260809 02:06:52.797794 24065 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39651 every 8 connection(s)
I20260809 02:06:52.798942 23995 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
I20260809 02:06:52.802532 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 23995
I20260809 02:06:52.802654 24066 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:06:52.802706 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786240941968412-1795-0/minicluster-data/master-0/wal/instance
I20260809 02:06:52.803045 1795 external_mini_cluster.cc:1428] Setting key f0f5ed0119efeff7e0f08f568ed0aa18
I20260809 02:06:52.808923 24069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:52.804522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35927 (local address 127.0.0.1:39651)
0809 02:06:52.804982 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0809 02:06:52.804997 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 02:06:52.805034 (+ 37us) server_negotiation.cc:407] Connection header received
0809 02:06:52.806078 (+ 1044us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:06:52.806101 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:06:52.806440 (+ 339us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:06:52.806862 (+ 422us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:06:52.807742 (+ 880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:06:52.807747 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:06:52.807758 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:06:52.807769 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:06:52.807824 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:06:52.807844 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:06:52.807995 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0809 02:06:52.808067 (+ 72us) server_negotiation.cc:299] Negotiation successful
0809 02:06:52.808224 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":312,"thread_start_us":142,"threads_started":1}
I20260809 02:06:52.809962 24066 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7a19f37c4474c34ba0540cbab499cc8: Bootstrap starting.
I20260809 02:06:52.811993 1795 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 02:06:52.812611 24066 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c7a19f37c4474c34ba0540cbab499cc8: Neither blocks nor log segments found. Creating new log.
I20260809 02:06:52.813390 24066 log.cc:824] T 00000000000000000000000000000000 P c7a19f37c4474c34ba0540cbab499cc8: Log is configured to *not* fsync() on all Append() calls
I20260809 02:06:52.815685 24066 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7a19f37c4474c34ba0540cbab499cc8: No bootstrap required, opened a new log
I20260809 02:06:52.816111 24069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:52.814317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56567 (local address 127.0.0.1:39651)
0809 02:06:52.814439 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0809 02:06:52.814445 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:06:52.814621 (+ 176us) server_negotiation.cc:407] Connection header received
0809 02:06:52.814811 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:06:52.814814 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:06:52.814854 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:06:52.814937 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:06:52.815710 (+ 773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:06:52.815714 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:06:52.815716 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:06:52.815729 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:06:52.815762 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:06:52.815777 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:06:52.815855 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 02:06:52.815945 (+ 90us) server_negotiation.cc:299] Negotiation successful
0809 02:06:52.815986 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 02:06:52.819599 24066 raft_consensus.cc:348] T 00000000000000000000000000000000 P c7a19f37c4474c34ba0540cbab499cc8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7a19f37c4474c34ba0540cbab499cc8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39651 } }
I20260809 02:06:52.819854 24066 raft_consensus.cc:374] T 00000000000000000000000000000000 P c7a19f37c4474c34ba0540cbab499cc8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:06:52.819888 24066 raft_consensus.cc:729] T 00000000000000000000000000000000 P c7a19f37c4474c34ba0540cbab499cc8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7a19f37c4474c34ba0540cbab499cc8, State: Initialized, Role: FOLLOWER
I20260809 02:06:52.820380 24066 consensus_queue.cc:260] T 00000000000000000000000000000000 P c7a19f37c4474c34ba0540cbab499cc8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7a19f37c4474c34ba0540cbab499cc8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39651 } }
I20260809 02:06:52.820547 24066 raft_consensus.cc:388] T 00000000000000000000000000000000 P c7a19f37c4474c34ba0540cbab499cc8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:06:52.820634 24066 raft_consensus.cc:482] T 00000000000000000000000000000000 P c7a19f37c4474c34ba0540cbab499cc8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:06:52.820750 24066 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c7a19f37c4474c34ba0540cbab499cc8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:06:52.821947 24066 raft_consensus.cc:504] T 00000000000000000000000000000000 P c7a19f37c4474c34ba0540cbab499cc8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7a19f37c4474c34ba0540cbab499cc8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39651 } }
I20260809 02:06:52.822327 24066 leader_election.cc:302] T 00000000000000000000000000000000 P c7a19f37c4474c34ba0540cbab499cc8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c7a19f37c4474c34ba0540cbab499cc8; no voters:
I20260809 02:06:52.822696 24066 leader_election.cc:288] T 00000000000000000000000000000000 P c7a19f37c4474c34ba0540cbab499cc8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:06:52.822836 24077 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c7a19f37c4474c34ba0540cbab499cc8 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:06:52.823138 24077 raft_consensus.cc:686] T 00000000000000000000000000000000 P c7a19f37c4474c34ba0540cbab499cc8 [term 1 LEADER]: Becoming Leader. State: Replica: c7a19f37c4474c34ba0540cbab499cc8, State: Running, Role: LEADER
I20260809 02:06:52.823478 24077 consensus_queue.cc:237] T 00000000000000000000000000000000 P c7a19f37c4474c34ba0540cbab499cc8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7a19f37c4474c34ba0540cbab499cc8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39651 } }
I20260809 02:06:52.823999 24066 sys_catalog.cc:564] T 00000000000000000000000000000000 P c7a19f37c4474c34ba0540cbab499cc8 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:06:52.825227 24079 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7a19f37c4474c34ba0540cbab499cc8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c7a19f37c4474c34ba0540cbab499cc8. Latest consensus state: current_term: 1 leader_uuid: "c7a19f37c4474c34ba0540cbab499cc8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7a19f37c4474c34ba0540cbab499cc8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39651 } } }
I20260809 02:06:52.825376 24079 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7a19f37c4474c34ba0540cbab499cc8 [sys.catalog]: This master's current role is: LEADER
I20260809 02:06:52.825733 24078 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7a19f37c4474c34ba0540cbab499cc8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c7a19f37c4474c34ba0540cbab499cc8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7a19f37c4474c34ba0540cbab499cc8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39651 } } }
I20260809 02:06:52.825845 24078 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7a19f37c4474c34ba0540cbab499cc8 [sys.catalog]: This master's current role is: LEADER
W20260809 02:06:52.828963 24093 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c7a19f37c4474c34ba0540cbab499cc8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 02:06:52.829041 24093 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 02:06:52.829130 24086 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:06:52.829941 24086 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:06:52.835723 24086 catalog_manager.cc:1347] Generated new cluster ID: aee7b16151d6455cae704b71e1af40d2
I20260809 02:06:52.835805 24086 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:06:52.865898 24086 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:06:52.866309 24086 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:06:52.886652 24086 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c7a19f37c4474c34ba0540cbab499cc8: Generated new TSK 0
I20260809 02:06:52.887544 24086 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:06:52.901465 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 23995
2026-08-09T02:06:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1559 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786240941968412-1795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:06:52 dist-test-slave-ss4n krb5kdc[24099](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:06:52 dist-test-slave-ss4n krb5kdc[24099](info): set up 2 sockets
Aug 09 02:06:52 dist-test-slave-ss4n krb5kdc[24099](info): commencing operation
krb5kdc: starting...
W20260809 02:06:54.061923 1795 mini_kdc.cc:121] Time spent starting KDC: real 1.138s user 0.006s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786240941968412-1795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786240941968412-1795-0/krb5kdc/test-user.keytab.
Aug 09 02:06:54 dist-test-slave-ss4n krb5kdc[24099](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241214, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:06:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:06:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786240941968412-1795-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786240941968412-1795-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 02:06:54.214444 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786240941968412-1795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786240941968412-1795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786240941968412-1795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45461
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786240941968412-1795-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45461
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:06:54.333359 24113 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:06:54.333720 24113 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:06:54.333765 24113 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:06:54.338060 24113 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:06:54.338145 24113 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:06:54.338165 24113 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:06:54.338183 24113 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:06:54.338198 24113 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:06:54.344354 24113 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786240941968412-1795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786240941968412-1795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45461
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786240941968412-1795-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45461
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24113
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786240941968412-1795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:06:54.345628 24113 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:06:54.346740 24113 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:06:54.356549 24122 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:06:54.356616 24113 server_base.cc:1034] running on GCE node
W20260809 02:06:54.356592 24119 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:06:54.356546 24120 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:06:54.357540 24113 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:06:54.358816 24113 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:06:54.360061 24113 hybrid_clock.cc:648] HybridClock initialized: now 1786241214360040 us; error 37 us; skew 500 ppm
Aug 09 02:06:54 dist-test-slave-ss4n krb5kdc[24099](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241214, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:06:54.363394 24113 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 02:06:54.364494 24113 webserver.cc:460] Webserver started at http://127.0.0.1:34171/ using document root <none> and password file <none>
I20260809 02:06:54.365151 24113 fs_manager.cc:359] Metadata directory not provided
I20260809 02:06:54.365257 24113 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:06:54.365586 24113 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:06:54.367887 24113 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786240941968412-1795-0/minicluster-data/master-0/data/instance:
uuid: "ccbd5714e2f6468683ff12fbd8eeb3a7"
format_stamp: "Formatted at 2026-08-09 02:06:54 on dist-test-slave-ss4n"
server_key: "6fa5dceb43172a71e8998309c68587ad"
server_key_iv: "376f6c46c267d1a4d253a1e1c43e5e18"
server_key_version: "encryptionkey@0"
I20260809 02:06:54.368577 24113 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786240941968412-1795-0/minicluster-data/master-0/wal/instance:
uuid: "ccbd5714e2f6468683ff12fbd8eeb3a7"
format_stamp: "Formatted at 2026-08-09 02:06:54 on dist-test-slave-ss4n"
server_key: "6fa5dceb43172a71e8998309c68587ad"
server_key_iv: "376f6c46c267d1a4d253a1e1c43e5e18"
server_key_version: "encryptionkey@0"
I20260809 02:06:54.373323 24113 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 02:06:54.376415 24129 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:06:54.377918 24113 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 02:06:54.378088 24113 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786240941968412-1795-0/minicluster-data/master-0/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786240941968412-1795-0/minicluster-data/master-0/wal
uuid: "ccbd5714e2f6468683ff12fbd8eeb3a7"
format_stamp: "Formatted at 2026-08-09 02:06:54 on dist-test-slave-ss4n"
server_key: "6fa5dceb43172a71e8998309c68587ad"
server_key_iv: "376f6c46c267d1a4d253a1e1c43e5e18"
server_key_version: "encryptionkey@0"
I20260809 02:06:54.378288 24113 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786240941968412-1795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786240941968412-1795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786240941968412-1795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:06:54.410060 24113 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:06:54.413962 24113 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:06:54.414222 24113 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:06:54.422798 24113 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45461
I20260809 02:06:54.422827 24182 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45461 every 8 connection(s)
I20260809 02:06:54.423949 24113 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
I20260809 02:06:54.427102 24184 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:06:54.433199 24184 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ccbd5714e2f6468683ff12fbd8eeb3a7: Bootstrap starting.
I20260809 02:06:54.433387 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 24113
I20260809 02:06:54.433583 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786240941968412-1795-0/minicluster-data/master-0/wal/instance
I20260809 02:06:54.433847 1795 external_mini_cluster.cc:1428] Setting key 458ff6c1693d005bc2b3a923ecafad87
I20260809 02:06:54.436282 24184 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ccbd5714e2f6468683ff12fbd8eeb3a7: Neither blocks nor log segments found. Creating new log.
I20260809 02:06:54.437357 24184 log.cc:824] T 00000000000000000000000000000000 P ccbd5714e2f6468683ff12fbd8eeb3a7: Log is configured to *not* fsync() on all Append() calls
I20260809 02:06:54.440035 24184 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ccbd5714e2f6468683ff12fbd8eeb3a7: No bootstrap required, opened a new log
Aug 09 02:06:54 dist-test-slave-ss4n krb5kdc[24099](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241214, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 02:06:54.444734 24184 raft_consensus.cc:348] T 00000000000000000000000000000000 P ccbd5714e2f6468683ff12fbd8eeb3a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccbd5714e2f6468683ff12fbd8eeb3a7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45461 } }
I20260809 02:06:54.445084 24184 raft_consensus.cc:374] T 00000000000000000000000000000000 P ccbd5714e2f6468683ff12fbd8eeb3a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:06:54.445147 24184 raft_consensus.cc:729] T 00000000000000000000000000000000 P ccbd5714e2f6468683ff12fbd8eeb3a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccbd5714e2f6468683ff12fbd8eeb3a7, State: Initialized, Role: FOLLOWER
I20260809 02:06:54.445722 24184 consensus_queue.cc:260] T 00000000000000000000000000000000 P ccbd5714e2f6468683ff12fbd8eeb3a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccbd5714e2f6468683ff12fbd8eeb3a7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45461 } }
I20260809 02:06:54.445904 24184 raft_consensus.cc:388] T 00000000000000000000000000000000 P ccbd5714e2f6468683ff12fbd8eeb3a7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:06:54.445992 24184 raft_consensus.cc:482] T 00000000000000000000000000000000 P ccbd5714e2f6468683ff12fbd8eeb3a7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:06:54.446098 24184 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ccbd5714e2f6468683ff12fbd8eeb3a7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:06:54.447570 24184 raft_consensus.cc:504] T 00000000000000000000000000000000 P ccbd5714e2f6468683ff12fbd8eeb3a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccbd5714e2f6468683ff12fbd8eeb3a7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45461 } }
I20260809 02:06:54.448014 24184 leader_election.cc:302] T 00000000000000000000000000000000 P ccbd5714e2f6468683ff12fbd8eeb3a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ccbd5714e2f6468683ff12fbd8eeb3a7; no voters:
I20260809 02:06:54.448493 24184 leader_election.cc:288] T 00000000000000000000000000000000 P ccbd5714e2f6468683ff12fbd8eeb3a7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:06:54.448724 24189 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ccbd5714e2f6468683ff12fbd8eeb3a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:06:54.449009 24189 raft_consensus.cc:686] T 00000000000000000000000000000000 P ccbd5714e2f6468683ff12fbd8eeb3a7 [term 1 LEADER]: Becoming Leader. State: Replica: ccbd5714e2f6468683ff12fbd8eeb3a7, State: Running, Role: LEADER
I20260809 02:06:54.449463 24189 consensus_queue.cc:237] T 00000000000000000000000000000000 P ccbd5714e2f6468683ff12fbd8eeb3a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccbd5714e2f6468683ff12fbd8eeb3a7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45461 } }
I20260809 02:06:54.450063 24184 sys_catalog.cc:564] T 00000000000000000000000000000000 P ccbd5714e2f6468683ff12fbd8eeb3a7 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:06:54.451503 24193 sys_catalog.cc:455] T 00000000000000000000000000000000 P ccbd5714e2f6468683ff12fbd8eeb3a7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ccbd5714e2f6468683ff12fbd8eeb3a7. Latest consensus state: current_term: 1 leader_uuid: "ccbd5714e2f6468683ff12fbd8eeb3a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccbd5714e2f6468683ff12fbd8eeb3a7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45461 } } }
I20260809 02:06:54.451671 24193 sys_catalog.cc:458] T 00000000000000000000000000000000 P ccbd5714e2f6468683ff12fbd8eeb3a7 [sys.catalog]: This master's current role is: LEADER
I20260809 02:06:54.452091 24191 sys_catalog.cc:455] T 00000000000000000000000000000000 P ccbd5714e2f6468683ff12fbd8eeb3a7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ccbd5714e2f6468683ff12fbd8eeb3a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccbd5714e2f6468683ff12fbd8eeb3a7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45461 } } }
I20260809 02:06:54.452212 24191 sys_catalog.cc:458] T 00000000000000000000000000000000 P ccbd5714e2f6468683ff12fbd8eeb3a7 [sys.catalog]: This master's current role is: LEADER
I20260809 02:06:54.452159 24204 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:06:54.452608 24187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:54.435375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50073 (local address 127.0.0.1:45461)
0809 02:06:54.435882 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0809 02:06:54.435901 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0809 02:06:54.435944 (+ 43us) server_negotiation.cc:407] Connection header received
0809 02:06:54.436936 (+ 992us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:06:54.436963 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:06:54.437451 (+ 488us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:06:54.437898 (+ 447us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:06:54.439069 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:54.440572 (+ 1503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:54.441541 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:54.442308 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:54.442489 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:54.445817 (+ 3328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:06:54.445824 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:06:54.445844 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:06:54.445888 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:06:54.448325 (+ 2437us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:06:54.449143 (+ 818us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:06:54.449151 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:06:54.449159 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:06:54.449347 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:06:54.449817 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:06:54.449820 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:06:54.449822 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:06:54.450455 (+ 633us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:06:54.450676 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0809 02:06:54.451046 (+ 370us) server_negotiation.cc:299] Negotiation successful
0809 02:06:54.451420 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":156,"threads_started":1}
I20260809 02:06:54.456260 24204 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:06:54.456302 1795 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 02:06:54.468568 24187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:54.458904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37161 (local address 127.0.0.1:45461)
0809 02:06:54.459061 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0809 02:06:54.459067 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:06:54.459347 (+ 280us) server_negotiation.cc:407] Connection header received
0809 02:06:54.459533 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:06:54.459537 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:06:54.459590 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:06:54.459688 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:06:54.460471 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:54.461160 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:54.462445 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:54.462741 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:54.462836 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:54.464627 (+ 1791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:06:54.464630 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:06:54.464633 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:06:54.464659 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:06:54.466434 (+ 1775us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:06:54.466981 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:06:54.466984 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:06:54.466986 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:06:54.467078 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:06:54.467464 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:06:54.467482 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:06:54.467486 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:06:54.467759 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:06:54.467922 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0809 02:06:54.468240 (+ 318us) server_negotiation.cc:299] Negotiation successful
0809 02:06:54.468382 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 02:06:54.469266 24204 catalog_manager.cc:1347] Generated new cluster ID: 9e0323d4520b4c2dbb2c39f44258fbbd
I20260809 02:06:54.469352 24204 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:06:54.479965 24204 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:06:54.481313 24204 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:06:54.495693 24204 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ccbd5714e2f6468683ff12fbd8eeb3a7: Generated new TSK 0
I20260809 02:06:54.496515 24204 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:06:54.508249 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 24113
2026-08-09T02:06:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1610 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786240941968412-1795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:06:54 dist-test-slave-ss4n krb5kdc[24221](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:06:54 dist-test-slave-ss4n krb5kdc[24221](info): set up 2 sockets
Aug 09 02:06:54 dist-test-slave-ss4n krb5kdc[24221](info): commencing operation
krb5kdc: starting...
W20260809 02:06:55.679961 1795 mini_kdc.cc:121] Time spent starting KDC: real 1.146s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786240941968412-1795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786240941968412-1795-0/krb5kdc/test-user.keytab.
Aug 09 02:06:55 dist-test-slave-ss4n krb5kdc[24221](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241215, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:06:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:06:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786240941968412-1795-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786240941968412-1795-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 02:06:55.834244 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786240941968412-1795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786240941968412-1795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786240941968412-1795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39963
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786240941968412-1795-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38971
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39963
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:06:55.952644 24235 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:06:55.952997 24235 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:06:55.953094 24235 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:06:55.957541 24235 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:06:55.957661 24235 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:06:55.957701 24235 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:06:55.957741 24235 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:06:55.957778 24235 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:06:55.963862 24235 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38971
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786240941968412-1795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786240941968412-1795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39963
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786240941968412-1795-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39963
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24235
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786240941968412-1795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:06:55.965104 24235 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:06:55.966215 24235 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:06:55.975430 24243 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:06:55.975440 24241 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:06:55.975458 24240 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:06:55.975710 24235 server_base.cc:1034] running on GCE node
I20260809 02:06:55.976250 24235 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:06:55.977418 24235 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:06:55.978644 24235 hybrid_clock.cc:648] HybridClock initialized: now 1786241215978616 us; error 45 us; skew 500 ppm
Aug 09 02:06:55 dist-test-slave-ss4n krb5kdc[24221](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241215, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:06:55.981918 24235 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 02:06:55.983002 24235 webserver.cc:460] Webserver started at http://127.0.0.1:33485/ using document root <none> and password file <none>
I20260809 02:06:55.983651 24235 fs_manager.cc:359] Metadata directory not provided
I20260809 02:06:55.983721 24235 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:06:55.984021 24235 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:06:55.986447 24235 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786240941968412-1795-0/minicluster-data/master-0/data/instance:
uuid: "bf8f0e4405e249d594f69376ae18a0e3"
format_stamp: "Formatted at 2026-08-09 02:06:55 on dist-test-slave-ss4n"
server_key: "1082e7613cb225635f5361467253a013"
server_key_iv: "51660e2c5a52125f5a3c50252c6a77ee"
server_key_version: "encryptionkey@0"
I20260809 02:06:55.987084 24235 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786240941968412-1795-0/minicluster-data/master-0/wal/instance:
uuid: "bf8f0e4405e249d594f69376ae18a0e3"
format_stamp: "Formatted at 2026-08-09 02:06:55 on dist-test-slave-ss4n"
server_key: "1082e7613cb225635f5361467253a013"
server_key_iv: "51660e2c5a52125f5a3c50252c6a77ee"
server_key_version: "encryptionkey@0"
I20260809 02:06:55.991243 24235 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 02:06:55.994119 24250 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:06:55.995474 24235 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 02:06:55.995675 24235 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786240941968412-1795-0/minicluster-data/master-0/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786240941968412-1795-0/minicluster-data/master-0/wal
uuid: "bf8f0e4405e249d594f69376ae18a0e3"
format_stamp: "Formatted at 2026-08-09 02:06:55 on dist-test-slave-ss4n"
server_key: "1082e7613cb225635f5361467253a013"
server_key_iv: "51660e2c5a52125f5a3c50252c6a77ee"
server_key_version: "encryptionkey@0"
I20260809 02:06:55.995795 24235 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786240941968412-1795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786240941968412-1795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786240941968412-1795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:06:56.013075 24235 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:06:56.017004 24235 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:06:56.017248 24235 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:06:56.025633 24235 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39963
I20260809 02:06:56.025679 24305 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39963 every 8 connection(s)
I20260809 02:06:56.026710 24235 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
I20260809 02:06:56.029690 24306 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:06:56.032981 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 24235
I20260809 02:06:56.033133 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786240941968412-1795-0/minicluster-data/master-0/wal/instance
I20260809 02:06:56.033595 1795 external_mini_cluster.cc:1428] Setting key 3aa8cd4b16980f4975794b6c58798a39
I20260809 02:06:56.035988 24306 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf8f0e4405e249d594f69376ae18a0e3: Bootstrap starting.
I20260809 02:06:56.038560 24306 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bf8f0e4405e249d594f69376ae18a0e3: Neither blocks nor log segments found. Creating new log.
I20260809 02:06:56.039446 24306 log.cc:824] T 00000000000000000000000000000000 P bf8f0e4405e249d594f69376ae18a0e3: Log is configured to *not* fsync() on all Append() calls
I20260809 02:06:56.041764 24306 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf8f0e4405e249d594f69376ae18a0e3: No bootstrap required, opened a new log
I20260809 02:06:56.044090 24309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:56.035222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35711 (local address 127.0.0.1:39963)
0809 02:06:56.035725 (+ 503us) server_negotiation.cc:206] Beginning negotiation
0809 02:06:56.035743 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0809 02:06:56.035782 (+ 39us) server_negotiation.cc:407] Connection header received
0809 02:06:56.036597 (+ 815us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:06:56.036615 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:06:56.036976 (+ 361us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:06:56.037283 (+ 307us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:06:56.037716 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:56.038985 (+ 1269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:56.040177 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:56.040721 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:56.040851 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:56.041805 (+ 954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:06:56.041837 (+ 32us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:06:56.041859 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:06:56.042339 (+ 480us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:06:56.042409 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:06:56.042435 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:06:56.042522 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0809 02:06:56.042705 (+ 183us) server_negotiation.cc:299] Negotiation successful
0809 02:06:56.042961 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":192,"threads_started":1}
I20260809 02:06:56.045146 24306 raft_consensus.cc:348] T 00000000000000000000000000000000 P bf8f0e4405e249d594f69376ae18a0e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf8f0e4405e249d594f69376ae18a0e3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39963 } }
I20260809 02:06:56.045428 24306 raft_consensus.cc:374] T 00000000000000000000000000000000 P bf8f0e4405e249d594f69376ae18a0e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:06:56.045501 24306 raft_consensus.cc:729] T 00000000000000000000000000000000 P bf8f0e4405e249d594f69376ae18a0e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf8f0e4405e249d594f69376ae18a0e3, State: Initialized, Role: FOLLOWER
I20260809 02:06:56.046020 24306 consensus_queue.cc:260] T 00000000000000000000000000000000 P bf8f0e4405e249d594f69376ae18a0e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf8f0e4405e249d594f69376ae18a0e3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39963 } }
I20260809 02:06:56.046206 24306 raft_consensus.cc:388] T 00000000000000000000000000000000 P bf8f0e4405e249d594f69376ae18a0e3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:06:56.046277 24306 raft_consensus.cc:482] T 00000000000000000000000000000000 P bf8f0e4405e249d594f69376ae18a0e3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:06:56.046373 24306 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bf8f0e4405e249d594f69376ae18a0e3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:06:56.047351 1795 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 02:06:56.047578 24306 raft_consensus.cc:504] T 00000000000000000000000000000000 P bf8f0e4405e249d594f69376ae18a0e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf8f0e4405e249d594f69376ae18a0e3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39963 } }
I20260809 02:06:56.047950 24306 leader_election.cc:302] T 00000000000000000000000000000000 P bf8f0e4405e249d594f69376ae18a0e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bf8f0e4405e249d594f69376ae18a0e3; no voters:
I20260809 02:06:56.048336 24306 leader_election.cc:288] T 00000000000000000000000000000000 P bf8f0e4405e249d594f69376ae18a0e3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:06:56.048491 24311 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bf8f0e4405e249d594f69376ae18a0e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:06:56.048776 24311 raft_consensus.cc:686] T 00000000000000000000000000000000 P bf8f0e4405e249d594f69376ae18a0e3 [term 1 LEADER]: Becoming Leader. State: Replica: bf8f0e4405e249d594f69376ae18a0e3, State: Running, Role: LEADER
I20260809 02:06:56.049189 24311 consensus_queue.cc:237] T 00000000000000000000000000000000 P bf8f0e4405e249d594f69376ae18a0e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf8f0e4405e249d594f69376ae18a0e3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39963 } }
I20260809 02:06:56.049758 24306 sys_catalog.cc:564] T 00000000000000000000000000000000 P bf8f0e4405e249d594f69376ae18a0e3 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:06:56.051445 24312 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf8f0e4405e249d594f69376ae18a0e3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bf8f0e4405e249d594f69376ae18a0e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf8f0e4405e249d594f69376ae18a0e3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39963 } } }
I20260809 02:06:56.051595 24312 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf8f0e4405e249d594f69376ae18a0e3 [sys.catalog]: This master's current role is: LEADER
I20260809 02:06:56.051964 24314 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf8f0e4405e249d594f69376ae18a0e3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bf8f0e4405e249d594f69376ae18a0e3. Latest consensus state: current_term: 1 leader_uuid: "bf8f0e4405e249d594f69376ae18a0e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf8f0e4405e249d594f69376ae18a0e3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39963 } } }
I20260809 02:06:56.052479 24314 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf8f0e4405e249d594f69376ae18a0e3 [sys.catalog]: This master's current role is: LEADER
I20260809 02:06:56.054944 24309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:56.049724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42753 (local address 127.0.0.1:39963)
0809 02:06:56.049857 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0809 02:06:56.049861 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:06:56.050146 (+ 285us) server_negotiation.cc:407] Connection header received
0809 02:06:56.050272 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:06:56.050276 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:06:56.050339 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:06:56.050446 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:06:56.050874 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:56.051810 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:56.053034 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:56.053303 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:56.053430 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:56.054424 (+ 994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:06:56.054428 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:06:56.054431 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:06:56.054445 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:06:56.054483 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:06:56.054495 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:06:56.054595 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0809 02:06:56.054698 (+ 103us) server_negotiation.cc:299] Negotiation successful
0809 02:06:56.054758 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
W20260809 02:06:56.055114 24332 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bf8f0e4405e249d594f69376ae18a0e3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 02:06:56.055191 24332 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 02:06:56.055289 24330 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:06:56.056160 24330 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:06:56.062224 24330 catalog_manager.cc:1347] Generated new cluster ID: 3c62b3f80362413fadf7b16e32d5d89e
I20260809 02:06:56.062314 24330 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:06:56.073055 24330 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:06:56.074103 24330 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:06:56.078022 24330 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bf8f0e4405e249d594f69376ae18a0e3: Generated new TSK 0
I20260809 02:06:56.078765 24330 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:06:56.083441 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 24235
2026-08-09T02:06:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1571 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786240941968412-1795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:06:56 dist-test-slave-ss4n krb5kdc[24339](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:06:56 dist-test-slave-ss4n krb5kdc[24339](info): set up 2 sockets
Aug 09 02:06:56 dist-test-slave-ss4n krb5kdc[24339](info): commencing operation
krb5kdc: starting...
W20260809 02:06:57.188266 1795 mini_kdc.cc:121] Time spent starting KDC: real 1.082s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786240941968412-1795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786240941968412-1795-0/krb5kdc/test-user.keytab.
Aug 09 02:06:57 dist-test-slave-ss4n krb5kdc[24339](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241217, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:06:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:06:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786240941968412-1795-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786240941968412-1795-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 02:06:57.330669 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786240941968412-1795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786240941968412-1795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786240941968412-1795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33721
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786240941968412-1795-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33721
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:06:57.452099 24353 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:06:57.452389 24353 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:06:57.452435 24353 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:06:57.456660 24353 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:06:57.456743 24353 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:06:57.456763 24353 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:06:57.456781 24353 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:06:57.456797 24353 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:06:57.462466 24353 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786240941968412-1795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786240941968412-1795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33721
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786240941968412-1795-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33721
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24353
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786240941968412-1795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:06:57.463624 24353 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:06:57.464717 24353 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:06:57.474064 24358 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:06:57.474236 24353 server_base.cc:1034] running on GCE node
W20260809 02:06:57.474279 24359 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:06:57.474061 24362 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:06:57.475066 24353 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:06:57.476274 24353 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:06:57.477438 24353 hybrid_clock.cc:648] HybridClock initialized: now 1786241217477387 us; error 66 us; skew 500 ppm
Aug 09 02:06:57 dist-test-slave-ss4n krb5kdc[24339](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241217, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:06:57.480536 24353 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 02:06:57.481657 24353 webserver.cc:460] Webserver started at http://127.0.0.1:33237/ using document root <none> and password file <none>
I20260809 02:06:57.482287 24353 fs_manager.cc:359] Metadata directory not provided
I20260809 02:06:57.482358 24353 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:06:57.482609 24353 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:06:57.485157 24353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786240941968412-1795-0/minicluster-data/master-0/data/instance:
uuid: "fd8919e799864e89a6e4283db9896392"
format_stamp: "Formatted at 2026-08-09 02:06:57 on dist-test-slave-ss4n"
server_key: "ed3a62fb90e91369987dee97c5833b37"
server_key_iv: "38c3363dac082e62bd6c2b9219d95b93"
server_key_version: "encryptionkey@0"
I20260809 02:06:57.486004 24353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786240941968412-1795-0/minicluster-data/master-0/wal/instance:
uuid: "fd8919e799864e89a6e4283db9896392"
format_stamp: "Formatted at 2026-08-09 02:06:57 on dist-test-slave-ss4n"
server_key: "ed3a62fb90e91369987dee97c5833b37"
server_key_iv: "38c3363dac082e62bd6c2b9219d95b93"
server_key_version: "encryptionkey@0"
I20260809 02:06:57.491953 24353 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.004s
I20260809 02:06:57.495441 24369 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:06:57.496973 24353 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 02:06:57.497143 24353 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786240941968412-1795-0/minicluster-data/master-0/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786240941968412-1795-0/minicluster-data/master-0/wal
uuid: "fd8919e799864e89a6e4283db9896392"
format_stamp: "Formatted at 2026-08-09 02:06:57 on dist-test-slave-ss4n"
server_key: "ed3a62fb90e91369987dee97c5833b37"
server_key_iv: "38c3363dac082e62bd6c2b9219d95b93"
server_key_version: "encryptionkey@0"
I20260809 02:06:57.497299 24353 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786240941968412-1795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786240941968412-1795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786240941968412-1795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:06:57.502481 24353 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:06:57.506301 24353 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:06:57.506525 24353 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:06:57.514389 24353 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33721
I20260809 02:06:57.514415 24424 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33721 every 8 connection(s)
I20260809 02:06:57.515458 24353 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
I20260809 02:06:57.518356 24425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:06:57.518818 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 24353
I20260809 02:06:57.518966 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786240941968412-1795-0/minicluster-data/master-0/wal/instance
I20260809 02:06:57.519295 1795 external_mini_cluster.cc:1428] Setting key c71048d1bac33943b257c4bdefa9111d
I20260809 02:06:57.525362 24428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:57.520901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41501 (local address 127.0.0.1:33721)
0809 02:06:57.521379 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0809 02:06:57.521400 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0809 02:06:57.521446 (+ 46us) server_negotiation.cc:407] Connection header received
0809 02:06:57.522334 (+ 888us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:06:57.522358 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:06:57.522720 (+ 362us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:06:57.523085 (+ 365us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:06:57.523953 (+ 868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:06:57.523959 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:06:57.523975 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:06:57.523987 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:06:57.524056 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:06:57.524083 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:06:57.524250 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0809 02:06:57.524320 (+ 70us) server_negotiation.cc:299] Negotiation successful
0809 02:06:57.524473 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":133,"threads_started":1}
I20260809 02:06:57.525434 24425 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd8919e799864e89a6e4283db9896392: Bootstrap starting.
I20260809 02:06:57.528076 24425 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fd8919e799864e89a6e4283db9896392: Neither blocks nor log segments found. Creating new log.
I20260809 02:06:57.528489 1795 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 02:06:57.528944 24425 log.cc:824] T 00000000000000000000000000000000 P fd8919e799864e89a6e4283db9896392: Log is configured to *not* fsync() on all Append() calls
I20260809 02:06:57.531162 24425 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd8919e799864e89a6e4283db9896392: No bootstrap required, opened a new log
I20260809 02:06:57.532846 24428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:57.530917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35081 (local address 127.0.0.1:33721)
0809 02:06:57.531063 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0809 02:06:57.531069 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:06:57.531321 (+ 252us) server_negotiation.cc:407] Connection header received
0809 02:06:57.531421 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:06:57.531423 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:06:57.531456 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:06:57.531536 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:06:57.532373 (+ 837us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:06:57.532376 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:06:57.532378 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:06:57.532389 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:06:57.532415 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:06:57.532426 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:06:57.532507 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 02:06:57.532629 (+ 122us) server_negotiation.cc:299] Negotiation successful
0809 02:06:57.532664 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 02:06:57.535561 24425 raft_consensus.cc:348] T 00000000000000000000000000000000 P fd8919e799864e89a6e4283db9896392 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd8919e799864e89a6e4283db9896392" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33721 } }
I20260809 02:06:57.535808 24425 raft_consensus.cc:374] T 00000000000000000000000000000000 P fd8919e799864e89a6e4283db9896392 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:06:57.535852 24425 raft_consensus.cc:729] T 00000000000000000000000000000000 P fd8919e799864e89a6e4283db9896392 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd8919e799864e89a6e4283db9896392, State: Initialized, Role: FOLLOWER
I20260809 02:06:57.536326 24425 consensus_queue.cc:260] T 00000000000000000000000000000000 P fd8919e799864e89a6e4283db9896392 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd8919e799864e89a6e4283db9896392" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33721 } }
I20260809 02:06:57.536484 24425 raft_consensus.cc:388] T 00000000000000000000000000000000 P fd8919e799864e89a6e4283db9896392 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:06:57.536548 24425 raft_consensus.cc:482] T 00000000000000000000000000000000 P fd8919e799864e89a6e4283db9896392 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:06:57.536657 24425 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fd8919e799864e89a6e4283db9896392 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:06:57.537842 24425 raft_consensus.cc:504] T 00000000000000000000000000000000 P fd8919e799864e89a6e4283db9896392 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd8919e799864e89a6e4283db9896392" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33721 } }
I20260809 02:06:57.538195 24425 leader_election.cc:302] T 00000000000000000000000000000000 P fd8919e799864e89a6e4283db9896392 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fd8919e799864e89a6e4283db9896392; no voters:
I20260809 02:06:57.538532 24425 leader_election.cc:288] T 00000000000000000000000000000000 P fd8919e799864e89a6e4283db9896392 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:06:57.538764 24435 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fd8919e799864e89a6e4283db9896392 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:06:57.539033 24435 raft_consensus.cc:686] T 00000000000000000000000000000000 P fd8919e799864e89a6e4283db9896392 [term 1 LEADER]: Becoming Leader. State: Replica: fd8919e799864e89a6e4283db9896392, State: Running, Role: LEADER
I20260809 02:06:57.539391 24435 consensus_queue.cc:237] T 00000000000000000000000000000000 P fd8919e799864e89a6e4283db9896392 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd8919e799864e89a6e4283db9896392" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33721 } }
I20260809 02:06:57.539990 24425 sys_catalog.cc:564] T 00000000000000000000000000000000 P fd8919e799864e89a6e4283db9896392 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:06:57.541357 24436 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd8919e799864e89a6e4283db9896392 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fd8919e799864e89a6e4283db9896392" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd8919e799864e89a6e4283db9896392" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33721 } } }
I20260809 02:06:57.541543 24436 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd8919e799864e89a6e4283db9896392 [sys.catalog]: This master's current role is: LEADER
I20260809 02:06:57.541590 24437 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd8919e799864e89a6e4283db9896392 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fd8919e799864e89a6e4283db9896392. Latest consensus state: current_term: 1 leader_uuid: "fd8919e799864e89a6e4283db9896392" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd8919e799864e89a6e4283db9896392" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33721 } } }
I20260809 02:06:57.541687 24437 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd8919e799864e89a6e4283db9896392 [sys.catalog]: This master's current role is: LEADER
W20260809 02:06:57.545320 24450 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fd8919e799864e89a6e4283db9896392: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 02:06:57.545629 24450 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 02:06:57.545727 24445 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:06:57.546586 24445 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:06:57.553608 24445 catalog_manager.cc:1347] Generated new cluster ID: 6313e894d5e147f0bfd49f78d5f7aa39
I20260809 02:06:57.553701 24445 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:06:57.567516 24445 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:06:57.567881 24445 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:06:57.575693 24445 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fd8919e799864e89a6e4283db9896392: Generated new TSK 0
I20260809 02:06:57.576510 24445 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:06:57.581955 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 24353
2026-08-09T02:06:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1498 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786240941968412-1795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:06:57 dist-test-slave-ss4n krb5kdc[24456](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:06:57 dist-test-slave-ss4n krb5kdc[24456](info): set up 2 sockets
Aug 09 02:06:57 dist-test-slave-ss4n krb5kdc[24456](info): commencing operation
krb5kdc: starting...
W20260809 02:06:58.767668 1795 mini_kdc.cc:121] Time spent starting KDC: real 1.163s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786240941968412-1795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786240941968412-1795-0/krb5kdc/test-user.keytab.
Aug 09 02:06:58 dist-test-slave-ss4n krb5kdc[24456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241218, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:06:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:06:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786240941968412-1795-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786240941968412-1795-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 02:06:58.920835 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786240941968412-1795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786240941968412-1795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786240941968412-1795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35617
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786240941968412-1795-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35617
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:06:59.036011 24469 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:06:59.036381 24469 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:06:59.036463 24469 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:06:59.040601 24469 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:06:59.040732 24469 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:06:59.040776 24469 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:06:59.040838 24469 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:06:59.040877 24469 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:06:59.046470 24469 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786240941968412-1795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786240941968412-1795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35617
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786240941968412-1795-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35617
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24469
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786240941968412-1795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:06:59.047642 24469 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:06:59.048728 24469 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:06:59.057799 24475 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:06:59.057821 24477 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:06:59.058178 24469 server_base.cc:1034] running on GCE node
W20260809 02:06:59.057799 24474 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:06:59.058785 24469 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:06:59.059926 24469 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:06:59.061123 24469 hybrid_clock.cc:648] HybridClock initialized: now 1786241219061105 us; error 37 us; skew 500 ppm
Aug 09 02:06:59 dist-test-slave-ss4n krb5kdc[24456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241219, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:06:59.069550 24469 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 02:06:59.070619 24469 webserver.cc:460] Webserver started at http://127.0.0.1:41355/ using document root <none> and password file <none>
I20260809 02:06:59.071170 24469 fs_manager.cc:359] Metadata directory not provided
I20260809 02:06:59.071233 24469 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:06:59.071403 24469 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:06:59.073540 24469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786240941968412-1795-0/minicluster-data/master-0/data/instance:
uuid: "6133950475db4f3cbb123e7481e11f97"
format_stamp: "Formatted at 2026-08-09 02:06:59 on dist-test-slave-ss4n"
server_key: "0b1b18b4ee4ef8077ac401f3256a2139"
server_key_iv: "c14c1511221cfc4d7760cb4dd80514dc"
server_key_version: "encryptionkey@0"
I20260809 02:06:59.074121 24469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786240941968412-1795-0/minicluster-data/master-0/wal/instance:
uuid: "6133950475db4f3cbb123e7481e11f97"
format_stamp: "Formatted at 2026-08-09 02:06:59 on dist-test-slave-ss4n"
server_key: "0b1b18b4ee4ef8077ac401f3256a2139"
server_key_iv: "c14c1511221cfc4d7760cb4dd80514dc"
server_key_version: "encryptionkey@0"
I20260809 02:06:59.077958 24469 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260809 02:06:59.080621 24485 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:06:59.081951 24469 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 02:06:59.082108 24469 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786240941968412-1795-0/minicluster-data/master-0/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786240941968412-1795-0/minicluster-data/master-0/wal
uuid: "6133950475db4f3cbb123e7481e11f97"
format_stamp: "Formatted at 2026-08-09 02:06:59 on dist-test-slave-ss4n"
server_key: "0b1b18b4ee4ef8077ac401f3256a2139"
server_key_iv: "c14c1511221cfc4d7760cb4dd80514dc"
server_key_version: "encryptionkey@0"
I20260809 02:06:59.082247 24469 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786240941968412-1795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786240941968412-1795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786240941968412-1795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:06:59.098378 24469 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:06:59.102138 24469 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:06:59.102388 24469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:06:59.110543 24469 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35617
I20260809 02:06:59.110567 24539 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35617 every 8 connection(s)
I20260809 02:06:59.111614 24469 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
I20260809 02:06:59.114619 24540 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:06:59.119128 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 24469
I20260809 02:06:59.119305 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786240941968412-1795-0/minicluster-data/master-0/wal/instance
I20260809 02:06:59.119611 1795 external_mini_cluster.cc:1428] Setting key 2131329ec464d22d50ee2bd90f400b13
I20260809 02:06:59.121279 24540 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6133950475db4f3cbb123e7481e11f97: Bootstrap starting.
I20260809 02:06:59.123822 24540 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6133950475db4f3cbb123e7481e11f97: Neither blocks nor log segments found. Creating new log.
I20260809 02:06:59.124859 24540 log.cc:824] T 00000000000000000000000000000000 P 6133950475db4f3cbb123e7481e11f97: Log is configured to *not* fsync() on all Append() calls
I20260809 02:06:59.127166 24540 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6133950475db4f3cbb123e7481e11f97: No bootstrap required, opened a new log
Aug 09 02:06:59 dist-test-slave-ss4n krb5kdc[24456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241218, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 02:06:59.131968 24540 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6133950475db4f3cbb123e7481e11f97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6133950475db4f3cbb123e7481e11f97" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35617 } }
I20260809 02:06:59.132290 24540 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6133950475db4f3cbb123e7481e11f97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:06:59.132369 24540 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6133950475db4f3cbb123e7481e11f97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6133950475db4f3cbb123e7481e11f97, State: Initialized, Role: FOLLOWER
I20260809 02:06:59.132887 24540 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6133950475db4f3cbb123e7481e11f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6133950475db4f3cbb123e7481e11f97" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35617 } }
I20260809 02:06:59.133083 24540 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6133950475db4f3cbb123e7481e11f97 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:06:59.133167 24540 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6133950475db4f3cbb123e7481e11f97 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:06:59.133278 24540 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6133950475db4f3cbb123e7481e11f97 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:06:59.134717 24540 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6133950475db4f3cbb123e7481e11f97 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6133950475db4f3cbb123e7481e11f97" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35617 } }
I20260809 02:06:59.135134 24540 leader_election.cc:302] T 00000000000000000000000000000000 P 6133950475db4f3cbb123e7481e11f97 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6133950475db4f3cbb123e7481e11f97; no voters:
I20260809 02:06:59.135525 24540 leader_election.cc:288] T 00000000000000000000000000000000 P 6133950475db4f3cbb123e7481e11f97 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:06:59.135625 24545 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6133950475db4f3cbb123e7481e11f97 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:06:59.135891 24545 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6133950475db4f3cbb123e7481e11f97 [term 1 LEADER]: Becoming Leader. State: Replica: 6133950475db4f3cbb123e7481e11f97, State: Running, Role: LEADER
I20260809 02:06:59.136207 24545 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6133950475db4f3cbb123e7481e11f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6133950475db4f3cbb123e7481e11f97" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35617 } }
I20260809 02:06:59.136276 24543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:59.121177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.245:45263 (local address 127.0.0.1:35617)
0809 02:06:59.121671 (+ 494us) server_negotiation.cc:206] Beginning negotiation
0809 02:06:59.121690 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0809 02:06:59.121728 (+ 38us) server_negotiation.cc:407] Connection header received
0809 02:06:59.122401 (+ 673us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:06:59.122418 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:06:59.122802 (+ 384us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:06:59.123153 (+ 351us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:06:59.124048 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:59.125673 (+ 1625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:59.126621 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:59.127367 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:59.127559 (+ 192us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:59.130225 (+ 2666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:06:59.130232 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:06:59.130254 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:06:59.130288 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:06:59.132472 (+ 2184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:06:59.133085 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:06:59.133093 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:06:59.133103 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:06:59.133266 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:06:59.133703 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:06:59.133709 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:06:59.133713 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:06:59.134295 (+ 582us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:06:59.134532 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0809 02:06:59.134821 (+ 289us) server_negotiation.cc:299] Negotiation successful
0809 02:06:59.135102 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":143,"threads_started":1}
I20260809 02:06:59.136958 24540 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6133950475db4f3cbb123e7481e11f97 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:06:59.137385 24545 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6133950475db4f3cbb123e7481e11f97 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6133950475db4f3cbb123e7481e11f97. Latest consensus state: current_term: 1 leader_uuid: "6133950475db4f3cbb123e7481e11f97" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6133950475db4f3cbb123e7481e11f97" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35617 } } }
I20260809 02:06:59.137540 24545 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6133950475db4f3cbb123e7481e11f97 [sys.catalog]: This master's current role is: LEADER
I20260809 02:06:59.137850 24546 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6133950475db4f3cbb123e7481e11f97 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6133950475db4f3cbb123e7481e11f97" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6133950475db4f3cbb123e7481e11f97" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35617 } } }
I20260809 02:06:59.137928 24546 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6133950475db4f3cbb123e7481e11f97 [sys.catalog]: This master's current role is: LEADER
I20260809 02:06:59.137959 24550 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:06:59.140077 1795 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 02:06:59.141074 24550 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:06:59.147557 24550 catalog_manager.cc:1347] Generated new cluster ID: c06dcafd7dc244488b11c60cfa3bc553
I20260809 02:06:59.147639 24550 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:06:59.154337 24543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:06:59.143875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.245:34901 (local address 127.0.0.1:35617)
0809 02:06:59.143988 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0809 02:06:59.144003 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 02:06:59.144234 (+ 231us) server_negotiation.cc:407] Connection header received
0809 02:06:59.144347 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:06:59.144351 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:06:59.144403 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:06:59.144501 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:06:59.145360 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:59.146229 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:59.147455 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:06:59.147820 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:06:59.147929 (+ 109us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:06:59.149964 (+ 2035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:06:59.149968 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:06:59.149971 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:06:59.149997 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:06:59.151958 (+ 1961us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:06:59.152546 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:06:59.152549 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:06:59.152551 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:06:59.152635 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:06:59.153016 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:06:59.153019 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:06:59.153021 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:06:59.153238 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:06:59.153379 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0809 02:06:59.154068 (+ 689us) server_negotiation.cc:299] Negotiation successful
0809 02:06:59.154165 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 02:06:59.163043 24550 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:06:59.164040 24550 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:06:59.172884 24550 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6133950475db4f3cbb123e7481e11f97: Generated new TSK 0
I20260809 02:06:59.173768 24550 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:06:59.181273 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 24469
W20260809 02:06:59.187865 24454 connection.cc:535] client connection to 127.0.0.1:35617 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35617): Connection reset by peer (error 104)
2026-08-09T02:06:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1599 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786240941968412-1795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:06:59 dist-test-slave-ss4n krb5kdc[24573](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:06:59 dist-test-slave-ss4n krb5kdc[24573](info): set up 2 sockets
Aug 09 02:06:59 dist-test-slave-ss4n krb5kdc[24573](info): commencing operation
krb5kdc: starting...
W20260809 02:07:00.327498 1795 mini_kdc.cc:121] Time spent starting KDC: real 1.124s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786240941968412-1795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786240941968412-1795-0/krb5kdc/test-user.keytab.
Aug 09 02:07:00 dist-test-slave-ss4n krb5kdc[24573](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241220, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:07:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:07:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786240941968412-1795-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786240941968412-1795-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 02:07:00.473388 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786240941968412-1795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786240941968412-1795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786240941968412-1795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45849
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786240941968412-1795-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45849
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:07:00.613534 24587 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:07:00.613859 24587 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:07:00.613905 24587 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:07:00.618158 24587 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:07:00.618234 24587 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:07:00.618254 24587 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:07:00.618271 24587 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:07:00.618287 24587 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:07:00.623822 24587 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786240941968412-1795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786240941968412-1795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45849
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786240941968412-1795-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45849
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24587
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786240941968412-1795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:07:00.625006 24587 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:07:00.626097 24587 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 02:07:00.634954 24587 server_base.cc:1034] running on GCE node
W20260809 02:07:00.634865 24594 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:07:00.634923 24593 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:07:00.634865 24596 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:07:00.635727 24587 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:07:00.636878 24587 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:07:00.638075 24587 hybrid_clock.cc:648] HybridClock initialized: now 1786241220638044 us; error 49 us; skew 500 ppm
Aug 09 02:07:00 dist-test-slave-ss4n krb5kdc[24573](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241220, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:07:00.641234 24587 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 02:07:00.642292 24587 webserver.cc:460] Webserver started at http://127.0.0.1:35755/ using document root <none> and password file <none>
I20260809 02:07:00.642948 24587 fs_manager.cc:359] Metadata directory not provided
I20260809 02:07:00.643051 24587 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:07:00.643275 24587 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:07:00.645589 24587 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786240941968412-1795-0/minicluster-data/master-0/data/instance:
uuid: "61c632ce25fc44a59227aaad0e5356df"
format_stamp: "Formatted at 2026-08-09 02:07:00 on dist-test-slave-ss4n"
server_key: "0d286bd65427b6621e86782109248943"
server_key_iv: "16646e0953b810ecbbd6ec83540e31ef"
server_key_version: "encryptionkey@0"
I20260809 02:07:00.646238 24587 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786240941968412-1795-0/minicluster-data/master-0/wal/instance:
uuid: "61c632ce25fc44a59227aaad0e5356df"
format_stamp: "Formatted at 2026-08-09 02:07:00 on dist-test-slave-ss4n"
server_key: "0d286bd65427b6621e86782109248943"
server_key_iv: "16646e0953b810ecbbd6ec83540e31ef"
server_key_version: "encryptionkey@0"
I20260809 02:07:00.650285 24587 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260809 02:07:00.653299 24605 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:07:00.654851 24587 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 02:07:00.655012 24587 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786240941968412-1795-0/minicluster-data/master-0/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786240941968412-1795-0/minicluster-data/master-0/wal
uuid: "61c632ce25fc44a59227aaad0e5356df"
format_stamp: "Formatted at 2026-08-09 02:07:00 on dist-test-slave-ss4n"
server_key: "0d286bd65427b6621e86782109248943"
server_key_iv: "16646e0953b810ecbbd6ec83540e31ef"
server_key_version: "encryptionkey@0"
I20260809 02:07:00.655171 24587 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786240941968412-1795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786240941968412-1795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786240941968412-1795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:07:00.675289 24587 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:07:00.679311 24587 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:07:00.679585 24587 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:07:00.687301 24587 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45849
I20260809 02:07:00.687331 24660 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45849 every 8 connection(s)
I20260809 02:07:00.688287 24587 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
I20260809 02:07:00.691349 24661 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:07:00.692395 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 24587
I20260809 02:07:00.692552 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786240941968412-1795-0/minicluster-data/master-0/wal/instance
I20260809 02:07:00.692893 1795 external_mini_cluster.cc:1428] Setting key 270241fc7e0d9c4834ac520b230ea369
I20260809 02:07:00.698339 24661 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61c632ce25fc44a59227aaad0e5356df: Bootstrap starting.
I20260809 02:07:00.700942 24661 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 61c632ce25fc44a59227aaad0e5356df: Neither blocks nor log segments found. Creating new log.
I20260809 02:07:00.701870 24661 log.cc:824] T 00000000000000000000000000000000 P 61c632ce25fc44a59227aaad0e5356df: Log is configured to *not* fsync() on all Append() calls
I20260809 02:07:00.703061 24664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:07:00.694537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.245:44177 (local address 127.0.0.1:45849)
0809 02:07:00.695096 (+ 559us) server_negotiation.cc:206] Beginning negotiation
0809 02:07:00.695112 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 02:07:00.695151 (+ 39us) server_negotiation.cc:407] Connection header received
0809 02:07:00.695818 (+ 667us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:07:00.695835 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:07:00.696206 (+ 371us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:07:00.696591 (+ 385us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:07:00.697089 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:07:00.698166 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:07:00.699197 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:07:00.699697 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:07:00.699886 (+ 189us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:07:00.700909 (+ 1023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:07:00.700913 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:07:00.700927 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:07:00.701400 (+ 473us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:07:00.701459 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:07:00.701556 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:07:00.701715 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0809 02:07:00.701908 (+ 193us) server_negotiation.cc:299] Negotiation successful
0809 02:07:00.702052 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":405,"thread_start_us":191,"threads_started":1}
I20260809 02:07:00.704574 24661 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61c632ce25fc44a59227aaad0e5356df: No bootstrap required, opened a new log
I20260809 02:07:00.706799 1795 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 02:07:00.709002 24661 raft_consensus.cc:348] T 00000000000000000000000000000000 P 61c632ce25fc44a59227aaad0e5356df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61c632ce25fc44a59227aaad0e5356df" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45849 } }
I20260809 02:07:00.709381 24661 raft_consensus.cc:374] T 00000000000000000000000000000000 P 61c632ce25fc44a59227aaad0e5356df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:07:00.709457 24661 raft_consensus.cc:729] T 00000000000000000000000000000000 P 61c632ce25fc44a59227aaad0e5356df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61c632ce25fc44a59227aaad0e5356df, State: Initialized, Role: FOLLOWER
I20260809 02:07:00.710106 24661 consensus_queue.cc:260] T 00000000000000000000000000000000 P 61c632ce25fc44a59227aaad0e5356df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61c632ce25fc44a59227aaad0e5356df" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45849 } }
I20260809 02:07:00.710305 24661 raft_consensus.cc:388] T 00000000000000000000000000000000 P 61c632ce25fc44a59227aaad0e5356df [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:07:00.710393 24661 raft_consensus.cc:482] T 00000000000000000000000000000000 P 61c632ce25fc44a59227aaad0e5356df [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:07:00.710533 24661 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 61c632ce25fc44a59227aaad0e5356df [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:07:00.711845 24661 raft_consensus.cc:504] T 00000000000000000000000000000000 P 61c632ce25fc44a59227aaad0e5356df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61c632ce25fc44a59227aaad0e5356df" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45849 } }
I20260809 02:07:00.712270 24661 leader_election.cc:302] T 00000000000000000000000000000000 P 61c632ce25fc44a59227aaad0e5356df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 61c632ce25fc44a59227aaad0e5356df; no voters:
I20260809 02:07:00.712787 24661 leader_election.cc:288] T 00000000000000000000000000000000 P 61c632ce25fc44a59227aaad0e5356df [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:07:00.712877 24672 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 61c632ce25fc44a59227aaad0e5356df [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:07:00.713109 24672 raft_consensus.cc:686] T 00000000000000000000000000000000 P 61c632ce25fc44a59227aaad0e5356df [term 1 LEADER]: Becoming Leader. State: Replica: 61c632ce25fc44a59227aaad0e5356df, State: Running, Role: LEADER
I20260809 02:07:00.713447 24672 consensus_queue.cc:237] T 00000000000000000000000000000000 P 61c632ce25fc44a59227aaad0e5356df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61c632ce25fc44a59227aaad0e5356df" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45849 } }
I20260809 02:07:00.714395 24661 sys_catalog.cc:564] T 00000000000000000000000000000000 P 61c632ce25fc44a59227aaad0e5356df [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:07:00.715257 24664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:07:00.709181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.245:33601 (local address 127.0.0.1:45849)
0809 02:07:00.709343 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0809 02:07:00.709348 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:07:00.709691 (+ 343us) server_negotiation.cc:407] Connection header received
0809 02:07:00.709875 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:07:00.709877 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:07:00.709926 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:07:00.710023 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:07:00.710479 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:07:00.711210 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:07:00.712621 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:07:00.712997 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:07:00.713168 (+ 171us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:07:00.714678 (+ 1510us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:07:00.714681 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:07:00.714684 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:07:00.714697 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:07:00.714728 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:07:00.714739 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:07:00.714862 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0809 02:07:00.715059 (+ 197us) server_negotiation.cc:299] Negotiation successful
0809 02:07:00.715099 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 02:07:00.715791 24673 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61c632ce25fc44a59227aaad0e5356df [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "61c632ce25fc44a59227aaad0e5356df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61c632ce25fc44a59227aaad0e5356df" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45849 } } }
I20260809 02:07:00.715942 24673 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61c632ce25fc44a59227aaad0e5356df [sys.catalog]: This master's current role is: LEADER
I20260809 02:07:00.716300 24674 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61c632ce25fc44a59227aaad0e5356df [sys.catalog]: SysCatalogTable state changed. Reason: New leader 61c632ce25fc44a59227aaad0e5356df. Latest consensus state: current_term: 1 leader_uuid: "61c632ce25fc44a59227aaad0e5356df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61c632ce25fc44a59227aaad0e5356df" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45849 } } }
I20260809 02:07:00.716420 24674 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61c632ce25fc44a59227aaad0e5356df [sys.catalog]: This master's current role is: LEADER
I20260809 02:07:00.716886 24682 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:07:00.721191 24682 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:07:00.727945 24682 catalog_manager.cc:1347] Generated new cluster ID: 8302e2956f974bb9a7a73b97630e0885
I20260809 02:07:00.728083 24682 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:07:00.755277 24682 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:07:00.756346 24682 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:07:00.761207 24682 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 61c632ce25fc44a59227aaad0e5356df: Generated new TSK 0
I20260809 02:07:00.762194 24682 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:07:00.774421 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 24587
W20260809 02:07:00.783978 24571 connection.cc:535] client connection to 127.0.0.1:45849 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45849): Connection reset by peer (error 104)
2026-08-09T02:07:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1595 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786240941968412-1795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:07:00 dist-test-slave-ss4n krb5kdc[24694](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:07:00 dist-test-slave-ss4n krb5kdc[24694](info): set up 2 sockets
Aug 09 02:07:00 dist-test-slave-ss4n krb5kdc[24694](info): commencing operation
krb5kdc: starting...
W20260809 02:07:01.962848 1795 mini_kdc.cc:121] Time spent starting KDC: real 1.163s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786240941968412-1795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786240941968412-1795-0/krb5kdc/test-user.keytab.
Aug 09 02:07:02 dist-test-slave-ss4n krb5kdc[24694](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241222, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:07:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:07:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786240941968412-1795-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786240941968412-1795-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 02:07:02.110399 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786240941968412-1795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786240941968412-1795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786240941968412-1795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45699
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786240941968412-1795-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45699
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:07:02.236222 24709 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:07:02.236552 24709 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:07:02.236598 24709 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:07:02.240835 24709 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:07:02.240916 24709 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:07:02.240935 24709 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:07:02.240952 24709 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:07:02.240968 24709 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:07:02.246564 24709 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786240941968412-1795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786240941968412-1795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45699
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786240941968412-1795-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45699
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24709
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786240941968412-1795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:07:02.247746 24709 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:07:02.248739 24709 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:07:02.257947 24717 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:07:02.257947 24714 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:07:02.258201 24709 server_base.cc:1034] running on GCE node
W20260809 02:07:02.258157 24715 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:07:02.258982 24709 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:07:02.260142 24709 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:07:02.261322 24709 hybrid_clock.cc:648] HybridClock initialized: now 1786241222261286 us; error 60 us; skew 500 ppm
Aug 09 02:07:02 dist-test-slave-ss4n krb5kdc[24694](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241222, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:07:02.264462 24709 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 02:07:02.265651 24709 webserver.cc:460] Webserver started at http://127.0.0.1:45751/ using document root <none> and password file <none>
I20260809 02:07:02.266250 24709 fs_manager.cc:359] Metadata directory not provided
I20260809 02:07:02.266314 24709 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:07:02.266539 24709 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:07:02.268787 24709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786240941968412-1795-0/minicluster-data/master-0/data/instance:
uuid: "626012eaa6664c208a7e84c05829cdf6"
format_stamp: "Formatted at 2026-08-09 02:07:02 on dist-test-slave-ss4n"
server_key: "c99fd8fbbd15a196819f51b2210cc57a"
server_key_iv: "28ec4747d51238821a48222947e1ec40"
server_key_version: "encryptionkey@0"
I20260809 02:07:02.269387 24709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786240941968412-1795-0/minicluster-data/master-0/wal/instance:
uuid: "626012eaa6664c208a7e84c05829cdf6"
format_stamp: "Formatted at 2026-08-09 02:07:02 on dist-test-slave-ss4n"
server_key: "c99fd8fbbd15a196819f51b2210cc57a"
server_key_iv: "28ec4747d51238821a48222947e1ec40"
server_key_version: "encryptionkey@0"
I20260809 02:07:02.273716 24709 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260809 02:07:02.276343 24724 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:07:02.277789 24709 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 02:07:02.277946 24709 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786240941968412-1795-0/minicluster-data/master-0/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786240941968412-1795-0/minicluster-data/master-0/wal
uuid: "626012eaa6664c208a7e84c05829cdf6"
format_stamp: "Formatted at 2026-08-09 02:07:02 on dist-test-slave-ss4n"
server_key: "c99fd8fbbd15a196819f51b2210cc57a"
server_key_iv: "28ec4747d51238821a48222947e1ec40"
server_key_version: "encryptionkey@0"
I20260809 02:07:02.278110 24709 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786240941968412-1795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786240941968412-1795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786240941968412-1795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:07:02.282989 24709 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:07:02.286480 24709 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:07:02.286653 24709 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:07:02.294638 24709 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45699
I20260809 02:07:02.294659 24778 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45699 every 8 connection(s)
I20260809 02:07:02.295691 24709 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
I20260809 02:07:02.298743 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 24709
I20260809 02:07:02.298905 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786240941968412-1795-0/minicluster-data/master-0/wal/instance
I20260809 02:07:02.299067 24779 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:07:02.299270 1795 external_mini_cluster.cc:1428] Setting key e3b5f2d1973f8bbcabb57b980b26ef50
I20260809 02:07:02.305645 24782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:07:02.300848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.245:46887 (local address 127.0.0.1:45699)
0809 02:07:02.301331 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0809 02:07:02.301352 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0809 02:07:02.301400 (+ 48us) server_negotiation.cc:407] Connection header received
0809 02:07:02.302388 (+ 988us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:07:02.302416 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:07:02.302842 (+ 426us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:07:02.303254 (+ 412us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:07:02.304043 (+ 789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:07:02.304049 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:07:02.304065 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:07:02.304078 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:07:02.304146 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:07:02.304175 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:07:02.304323 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0809 02:07:02.304395 (+ 72us) server_negotiation.cc:299] Negotiation successful
0809 02:07:02.304566 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":131,"threads_started":1}
I20260809 02:07:02.307467 24779 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 626012eaa6664c208a7e84c05829cdf6: Bootstrap starting.
I20260809 02:07:02.308900 1795 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 02:07:02.310377 24779 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 626012eaa6664c208a7e84c05829cdf6: Neither blocks nor log segments found. Creating new log.
I20260809 02:07:02.311293 24779 log.cc:824] T 00000000000000000000000000000000 P 626012eaa6664c208a7e84c05829cdf6: Log is configured to *not* fsync() on all Append() calls
I20260809 02:07:02.313109 24782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:07:02.311224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.245:40955 (local address 127.0.0.1:45699)
0809 02:07:02.311385 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0809 02:07:02.311391 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:07:02.311601 (+ 210us) server_negotiation.cc:407] Connection header received
0809 02:07:02.311770 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:07:02.311773 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:07:02.311815 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:07:02.311903 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:07:02.312695 (+ 792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:07:02.312698 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:07:02.312701 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:07:02.312712 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:07:02.312741 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:07:02.312758 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:07:02.312861 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0809 02:07:02.312939 (+ 78us) server_negotiation.cc:299] Negotiation successful
0809 02:07:02.312983 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 02:07:02.313833 24779 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 626012eaa6664c208a7e84c05829cdf6: No bootstrap required, opened a new log
I20260809 02:07:02.318367 24779 raft_consensus.cc:348] T 00000000000000000000000000000000 P 626012eaa6664c208a7e84c05829cdf6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "626012eaa6664c208a7e84c05829cdf6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45699 } }
I20260809 02:07:02.318658 24779 raft_consensus.cc:374] T 00000000000000000000000000000000 P 626012eaa6664c208a7e84c05829cdf6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:07:02.318745 24779 raft_consensus.cc:729] T 00000000000000000000000000000000 P 626012eaa6664c208a7e84c05829cdf6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 626012eaa6664c208a7e84c05829cdf6, State: Initialized, Role: FOLLOWER
I20260809 02:07:02.319250 24779 consensus_queue.cc:260] T 00000000000000000000000000000000 P 626012eaa6664c208a7e84c05829cdf6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "626012eaa6664c208a7e84c05829cdf6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45699 } }
I20260809 02:07:02.319389 24779 raft_consensus.cc:388] T 00000000000000000000000000000000 P 626012eaa6664c208a7e84c05829cdf6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:07:02.319489 24779 raft_consensus.cc:482] T 00000000000000000000000000000000 P 626012eaa6664c208a7e84c05829cdf6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:07:02.319630 24779 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 626012eaa6664c208a7e84c05829cdf6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:07:02.320771 24779 raft_consensus.cc:504] T 00000000000000000000000000000000 P 626012eaa6664c208a7e84c05829cdf6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "626012eaa6664c208a7e84c05829cdf6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45699 } }
I20260809 02:07:02.321182 24779 leader_election.cc:302] T 00000000000000000000000000000000 P 626012eaa6664c208a7e84c05829cdf6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 626012eaa6664c208a7e84c05829cdf6; no voters:
I20260809 02:07:02.321620 24779 leader_election.cc:288] T 00000000000000000000000000000000 P 626012eaa6664c208a7e84c05829cdf6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:07:02.321748 24791 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 626012eaa6664c208a7e84c05829cdf6 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:07:02.322046 24791 raft_consensus.cc:686] T 00000000000000000000000000000000 P 626012eaa6664c208a7e84c05829cdf6 [term 1 LEADER]: Becoming Leader. State: Replica: 626012eaa6664c208a7e84c05829cdf6, State: Running, Role: LEADER
I20260809 02:07:02.322451 24791 consensus_queue.cc:237] T 00000000000000000000000000000000 P 626012eaa6664c208a7e84c05829cdf6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "626012eaa6664c208a7e84c05829cdf6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45699 } }
I20260809 02:07:02.323246 24779 sys_catalog.cc:564] T 00000000000000000000000000000000 P 626012eaa6664c208a7e84c05829cdf6 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:07:02.324498 24792 sys_catalog.cc:455] T 00000000000000000000000000000000 P 626012eaa6664c208a7e84c05829cdf6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "626012eaa6664c208a7e84c05829cdf6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "626012eaa6664c208a7e84c05829cdf6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45699 } } }
I20260809 02:07:02.324543 24793 sys_catalog.cc:455] T 00000000000000000000000000000000 P 626012eaa6664c208a7e84c05829cdf6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 626012eaa6664c208a7e84c05829cdf6. Latest consensus state: current_term: 1 leader_uuid: "626012eaa6664c208a7e84c05829cdf6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "626012eaa6664c208a7e84c05829cdf6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45699 } } }
I20260809 02:07:02.324685 24793 sys_catalog.cc:458] T 00000000000000000000000000000000 P 626012eaa6664c208a7e84c05829cdf6 [sys.catalog]: This master's current role is: LEADER
I20260809 02:07:02.324687 24792 sys_catalog.cc:458] T 00000000000000000000000000000000 P 626012eaa6664c208a7e84c05829cdf6 [sys.catalog]: This master's current role is: LEADER
I20260809 02:07:02.325237 24801 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:07:02.328140 24801 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:07:02.334894 24801 catalog_manager.cc:1347] Generated new cluster ID: b2c1e9322b114d33a4047126246a004d
I20260809 02:07:02.335016 24801 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:07:02.357072 24801 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:07:02.357467 24801 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:07:02.371937 24801 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 626012eaa6664c208a7e84c05829cdf6: Generated new TSK 0
I20260809 02:07:02.372849 24801 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:07:02.384570 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 24709
2026-08-09T02:07:02Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1608 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786240941968412-1795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:07:02 dist-test-slave-ss4n krb5kdc[24814](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:07:02 dist-test-slave-ss4n krb5kdc[24814](info): set up 2 sockets
Aug 09 02:07:02 dist-test-slave-ss4n krb5kdc[24814](info): commencing operation
krb5kdc: starting...
W20260809 02:07:03.492579 1795 mini_kdc.cc:121] Time spent starting KDC: real 1.084s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786240941968412-1795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786240941968412-1795-0/krb5kdc/test-user.keytab.
Aug 09 02:07:03 dist-test-slave-ss4n krb5kdc[24814](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241223, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:07:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:07:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786240941968412-1795-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786240941968412-1795-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 02:07:03.634030 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786240941968412-1795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786240941968412-1795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786240941968412-1795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35753
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786240941968412-1795-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35753
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:07:03.761884 24827 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:07:03.762269 24827 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:07:03.762362 24827 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:07:03.767055 24827 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:07:03.767189 24827 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:07:03.767228 24827 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:07:03.767268 24827 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:07:03.767299 24827 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:07:03.773005 24827 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786240941968412-1795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786240941968412-1795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35753
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786240941968412-1795-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35753
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24827
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786240941968412-1795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:07:03.774230 24827 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:07:03.775336 24827 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:07:03.784871 24832 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:07:03.784891 24835 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:07:03.784889 24833 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:07:03.785199 24827 server_base.cc:1034] running on GCE node
I20260809 02:07:03.786046 24827 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:07:03.787220 24827 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:07:03.788422 24827 hybrid_clock.cc:648] HybridClock initialized: now 1786241223788389 us; error 52 us; skew 500 ppm
Aug 09 02:07:03 dist-test-slave-ss4n krb5kdc[24814](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241223, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:07:03.791657 24827 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 02:07:03.792687 24827 webserver.cc:460] Webserver started at http://127.0.0.1:41243/ using document root <none> and password file <none>
I20260809 02:07:03.793284 24827 fs_manager.cc:359] Metadata directory not provided
I20260809 02:07:03.793411 24827 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:07:03.793710 24827 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:07:03.795873 24827 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786240941968412-1795-0/minicluster-data/master-0/data/instance:
uuid: "6b376e42285e4faea076a0d89f5e79b3"
format_stamp: "Formatted at 2026-08-09 02:07:03 on dist-test-slave-ss4n"
server_key: "21935b4981f7cafcb8ec3cc179851241"
server_key_iv: "e2807a4d6e7d31a97471c47acd588db3"
server_key_version: "encryptionkey@0"
I20260809 02:07:03.796509 24827 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786240941968412-1795-0/minicluster-data/master-0/wal/instance:
uuid: "6b376e42285e4faea076a0d89f5e79b3"
format_stamp: "Formatted at 2026-08-09 02:07:03 on dist-test-slave-ss4n"
server_key: "21935b4981f7cafcb8ec3cc179851241"
server_key_iv: "e2807a4d6e7d31a97471c47acd588db3"
server_key_version: "encryptionkey@0"
I20260809 02:07:03.800576 24827 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 02:07:03.803418 24842 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:07:03.804773 24827 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 02:07:03.804935 24827 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786240941968412-1795-0/minicluster-data/master-0/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786240941968412-1795-0/minicluster-data/master-0/wal
uuid: "6b376e42285e4faea076a0d89f5e79b3"
format_stamp: "Formatted at 2026-08-09 02:07:03 on dist-test-slave-ss4n"
server_key: "21935b4981f7cafcb8ec3cc179851241"
server_key_iv: "e2807a4d6e7d31a97471c47acd588db3"
server_key_version: "encryptionkey@0"
I20260809 02:07:03.805073 24827 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786240941968412-1795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786240941968412-1795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786240941968412-1795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:07:03.825186 24827 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:07:03.829160 24827 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:07:03.829417 24827 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:07:03.837826 24827 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35753
I20260809 02:07:03.837877 24896 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35753 every 8 connection(s)
I20260809 02:07:03.838896 24827 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
I20260809 02:07:03.841874 24897 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:07:03.842638 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 24827
I20260809 02:07:03.842806 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786240941968412-1795-0/minicluster-data/master-0/wal/instance
I20260809 02:07:03.843144 1795 external_mini_cluster.cc:1428] Setting key 0bb97163abdde0d692c616eb53af386b
I20260809 02:07:03.850214 24897 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b376e42285e4faea076a0d89f5e79b3: Bootstrap starting.
I20260809 02:07:03.853087 24897 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6b376e42285e4faea076a0d89f5e79b3: Neither blocks nor log segments found. Creating new log.
I20260809 02:07:03.853993 24897 log.cc:824] T 00000000000000000000000000000000 P 6b376e42285e4faea076a0d89f5e79b3: Log is configured to *not* fsync() on all Append() calls
Aug 09 02:07:03 dist-test-slave-ss4n krb5kdc[24814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241223, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 02:07:03.856556 24897 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b376e42285e4faea076a0d89f5e79b3: No bootstrap required, opened a new log
I20260809 02:07:03.861704 24900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:07:03.844790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.245:33583 (local address 127.0.0.1:35753)
0809 02:07:03.845505 (+ 715us) server_negotiation.cc:206] Beginning negotiation
0809 02:07:03.845536 (+ 31us) server_negotiation.cc:399] Waiting for connection header
0809 02:07:03.845601 (+ 65us) server_negotiation.cc:407] Connection header received
0809 02:07:03.846576 (+ 975us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:07:03.846604 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:07:03.847149 (+ 545us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:07:03.847575 (+ 426us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:07:03.849047 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:07:03.850670 (+ 1623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:07:03.851648 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:07:03.852397 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:07:03.852578 (+ 181us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:07:03.855609 (+ 3031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:07:03.855617 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:07:03.855636 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:07:03.855669 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:07:03.858023 (+ 2354us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:07:03.858607 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:07:03.858614 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:07:03.858621 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:07:03.858776 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:07:03.859134 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:07:03.859140 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:07:03.859143 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:07:03.859777 (+ 634us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:07:03.860003 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0809 02:07:03.860255 (+ 252us) server_negotiation.cc:299] Negotiation successful
0809 02:07:03.860519 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390,"thread_start_us":172,"threads_started":1}
I20260809 02:07:03.861938 24897 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6b376e42285e4faea076a0d89f5e79b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b376e42285e4faea076a0d89f5e79b3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35753 } }
I20260809 02:07:03.862278 24897 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6b376e42285e4faea076a0d89f5e79b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:07:03.862368 24897 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6b376e42285e4faea076a0d89f5e79b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b376e42285e4faea076a0d89f5e79b3, State: Initialized, Role: FOLLOWER
I20260809 02:07:03.862927 24897 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6b376e42285e4faea076a0d89f5e79b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b376e42285e4faea076a0d89f5e79b3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35753 } }
I20260809 02:07:03.863133 24897 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6b376e42285e4faea076a0d89f5e79b3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:07:03.863210 24897 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6b376e42285e4faea076a0d89f5e79b3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:07:03.863327 24897 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6b376e42285e4faea076a0d89f5e79b3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:07:03.864698 24897 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6b376e42285e4faea076a0d89f5e79b3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b376e42285e4faea076a0d89f5e79b3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35753 } }
I20260809 02:07:03.865126 24897 leader_election.cc:302] T 00000000000000000000000000000000 P 6b376e42285e4faea076a0d89f5e79b3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6b376e42285e4faea076a0d89f5e79b3; no voters:
I20260809 02:07:03.865527 1795 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 02:07:03.865568 24897 leader_election.cc:288] T 00000000000000000000000000000000 P 6b376e42285e4faea076a0d89f5e79b3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:07:03.865721 24902 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6b376e42285e4faea076a0d89f5e79b3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:07:03.865950 24902 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6b376e42285e4faea076a0d89f5e79b3 [term 1 LEADER]: Becoming Leader. State: Replica: 6b376e42285e4faea076a0d89f5e79b3, State: Running, Role: LEADER
I20260809 02:07:03.866361 24902 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6b376e42285e4faea076a0d89f5e79b3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b376e42285e4faea076a0d89f5e79b3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35753 } }
I20260809 02:07:03.867086 24897 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6b376e42285e4faea076a0d89f5e79b3 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:07:03.868124 24903 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b376e42285e4faea076a0d89f5e79b3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6b376e42285e4faea076a0d89f5e79b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b376e42285e4faea076a0d89f5e79b3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35753 } } }
I20260809 02:07:03.868257 24903 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b376e42285e4faea076a0d89f5e79b3 [sys.catalog]: This master's current role is: LEADER
I20260809 02:07:03.868501 24903 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b376e42285e4faea076a0d89f5e79b3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6b376e42285e4faea076a0d89f5e79b3. Latest consensus state: current_term: 1 leader_uuid: "6b376e42285e4faea076a0d89f5e79b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b376e42285e4faea076a0d89f5e79b3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35753 } } }
I20260809 02:07:03.868587 24903 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b376e42285e4faea076a0d89f5e79b3 [sys.catalog]: This master's current role is: LEADER
W20260809 02:07:03.873243 24923 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6b376e42285e4faea076a0d89f5e79b3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 02:07:03.873404 24923 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 02:07:03.873549 24915 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:07:03.874524 24915 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:07:03.879729 24900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:07:03.868059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.245:45433 (local address 127.0.0.1:35753)
0809 02:07:03.868194 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0809 02:07:03.868199 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:07:03.869892 (+ 1693us) server_negotiation.cc:407] Connection header received
0809 02:07:03.870109 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:07:03.870112 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:07:03.870163 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:07:03.870249 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:07:03.871209 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:07:03.871854 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:07:03.872840 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:07:03.873188 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:07:03.873324 (+ 136us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:07:03.875133 (+ 1809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:07:03.875136 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:07:03.875139 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:07:03.875178 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:07:03.876991 (+ 1813us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:07:03.877712 (+ 721us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:07:03.877716 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:07:03.877718 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:07:03.877816 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:07:03.878246 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:07:03.878249 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:07:03.878252 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:07:03.878485 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:07:03.878632 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0809 02:07:03.879417 (+ 785us) server_negotiation.cc:299] Negotiation successful
0809 02:07:03.879558 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 02:07:03.881606 24915 catalog_manager.cc:1347] Generated new cluster ID: 6c823dd0fbf64f5eaea4acebe322acce
I20260809 02:07:03.881697 24915 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:07:03.896906 24915 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:07:03.898110 24915 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:07:03.906973 24915 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6b376e42285e4faea076a0d89f5e79b3: Generated new TSK 0
I20260809 02:07:03.907879 24915 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:07:03.920902 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 24827
W20260809 02:07:03.929219 24812 connection.cc:535] client connection to 127.0.0.1:35753 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35753): Connection reset by peer (error 104)
2026-08-09T02:07:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1536 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786240941968412-1795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:07:03 dist-test-slave-ss4n krb5kdc[24929](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:07:03 dist-test-slave-ss4n krb5kdc[24929](info): set up 2 sockets
Aug 09 02:07:03 dist-test-slave-ss4n krb5kdc[24929](info): commencing operation
krb5kdc: starting...
W20260809 02:07:05.068984 1795 mini_kdc.cc:121] Time spent starting KDC: real 1.125s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786240941968412-1795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786240941968412-1795-0/krb5kdc/test-user.keytab.
Aug 09 02:07:05 dist-test-slave-ss4n krb5kdc[24929](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241225, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:07:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:07:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786240941968412-1795-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786240941968412-1795-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 02:07:05.214902 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786240941968412-1795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786240941968412-1795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786240941968412-1795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40901
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786240941968412-1795-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40901
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:07:05.334597 24943 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:07:05.334955 24943 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:07:05.335045 24943 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:07:05.339376 24943 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:07:05.339488 24943 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:07:05.339529 24943 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:07:05.339586 24943 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:07:05.339649 24943 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:07:05.345337 24943 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786240941968412-1795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786240941968412-1795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40901
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786240941968412-1795-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40901
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24943
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786240941968412-1795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:07:05.346623 24943 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:07:05.347751 24943 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:07:05.356930 24948 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:07:05.356930 24951 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:07:05.356935 24949 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:07:05.357795 24943 server_base.cc:1034] running on GCE node
I20260809 02:07:05.358438 24943 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:07:05.359659 24943 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:07:05.360832 24943 hybrid_clock.cc:648] HybridClock initialized: now 1786241225360805 us; error 44 us; skew 500 ppm
Aug 09 02:07:05 dist-test-slave-ss4n krb5kdc[24929](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241225, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:07:05.368153 24943 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 02:07:05.369455 24943 webserver.cc:460] Webserver started at http://127.0.0.1:37563/ using document root <none> and password file <none>
I20260809 02:07:05.370173 24943 fs_manager.cc:359] Metadata directory not provided
I20260809 02:07:05.370287 24943 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:07:05.370553 24943 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:07:05.372987 24943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786240941968412-1795-0/minicluster-data/master-0/data/instance:
uuid: "404a130a1eb34246a468c5d840b4ef0b"
format_stamp: "Formatted at 2026-08-09 02:07:05 on dist-test-slave-ss4n"
server_key: "a7c713179b8f8d58eb0eb2b5a0a20841"
server_key_iv: "9db4b9fb7fbbb1803057893ed0aa041c"
server_key_version: "encryptionkey@0"
I20260809 02:07:05.373725 24943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786240941968412-1795-0/minicluster-data/master-0/wal/instance:
uuid: "404a130a1eb34246a468c5d840b4ef0b"
format_stamp: "Formatted at 2026-08-09 02:07:05 on dist-test-slave-ss4n"
server_key: "a7c713179b8f8d58eb0eb2b5a0a20841"
server_key_iv: "9db4b9fb7fbbb1803057893ed0aa041c"
server_key_version: "encryptionkey@0"
I20260809 02:07:05.378103 24943 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 02:07:05.380860 24959 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:07:05.382589 24943 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 02:07:05.382756 24943 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786240941968412-1795-0/minicluster-data/master-0/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786240941968412-1795-0/minicluster-data/master-0/wal
uuid: "404a130a1eb34246a468c5d840b4ef0b"
format_stamp: "Formatted at 2026-08-09 02:07:05 on dist-test-slave-ss4n"
server_key: "a7c713179b8f8d58eb0eb2b5a0a20841"
server_key_iv: "9db4b9fb7fbbb1803057893ed0aa041c"
server_key_version: "encryptionkey@0"
I20260809 02:07:05.382903 24943 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786240941968412-1795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786240941968412-1795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786240941968412-1795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:07:05.413801 24943 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:07:05.417254 24943 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:07:05.417575 24943 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:07:05.426659 24943 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40901
I20260809 02:07:05.426702 25013 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40901 every 8 connection(s)
I20260809 02:07:05.427783 24943 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
I20260809 02:07:05.430925 25014 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:07:05.433548 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 24943
I20260809 02:07:05.433758 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786240941968412-1795-0/minicluster-data/master-0/wal/instance
I20260809 02:07:05.434072 1795 external_mini_cluster.cc:1428] Setting key 8ded393db1a5a772c124989f8a88226b
I20260809 02:07:05.437284 25014 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 404a130a1eb34246a468c5d840b4ef0b: Bootstrap starting.
I20260809 02:07:05.440189 25014 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 404a130a1eb34246a468c5d840b4ef0b: Neither blocks nor log segments found. Creating new log.
I20260809 02:07:05.441171 25014 log.cc:824] T 00000000000000000000000000000000 P 404a130a1eb34246a468c5d840b4ef0b: Log is configured to *not* fsync() on all Append() calls
I20260809 02:07:05.443825 25014 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 404a130a1eb34246a468c5d840b4ef0b: No bootstrap required, opened a new log
I20260809 02:07:05.444156 25017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:07:05.435528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.245:58709 (local address 127.0.0.1:40901)
0809 02:07:05.435989 (+ 461us) server_negotiation.cc:206] Beginning negotiation
0809 02:07:05.436005 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 02:07:05.436043 (+ 38us) server_negotiation.cc:407] Connection header received
0809 02:07:05.436723 (+ 680us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:07:05.436742 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:07:05.437110 (+ 368us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:07:05.437444 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:07:05.437848 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:07:05.438862 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:07:05.440178 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:07:05.440769 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:07:05.440908 (+ 139us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:07:05.442067 (+ 1159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:07:05.442075 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:07:05.442098 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:07:05.442622 (+ 524us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:07:05.442684 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:07:05.442709 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:07:05.442827 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0809 02:07:05.443104 (+ 277us) server_negotiation.cc:299] Negotiation successful
0809 02:07:05.443280 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":148,"threads_started":1}
I20260809 02:07:05.447706 1795 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 02:07:05.448513 25014 raft_consensus.cc:348] T 00000000000000000000000000000000 P 404a130a1eb34246a468c5d840b4ef0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "404a130a1eb34246a468c5d840b4ef0b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40901 } }
I20260809 02:07:05.448776 25014 raft_consensus.cc:374] T 00000000000000000000000000000000 P 404a130a1eb34246a468c5d840b4ef0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:07:05.448849 25014 raft_consensus.cc:729] T 00000000000000000000000000000000 P 404a130a1eb34246a468c5d840b4ef0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 404a130a1eb34246a468c5d840b4ef0b, State: Initialized, Role: FOLLOWER
I20260809 02:07:05.449427 25014 consensus_queue.cc:260] T 00000000000000000000000000000000 P 404a130a1eb34246a468c5d840b4ef0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "404a130a1eb34246a468c5d840b4ef0b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40901 } }
I20260809 02:07:05.449635 25014 raft_consensus.cc:388] T 00000000000000000000000000000000 P 404a130a1eb34246a468c5d840b4ef0b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:07:05.449734 25014 raft_consensus.cc:482] T 00000000000000000000000000000000 P 404a130a1eb34246a468c5d840b4ef0b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:07:05.449862 25014 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 404a130a1eb34246a468c5d840b4ef0b [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:07:05.451346 25014 raft_consensus.cc:504] T 00000000000000000000000000000000 P 404a130a1eb34246a468c5d840b4ef0b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "404a130a1eb34246a468c5d840b4ef0b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40901 } }
I20260809 02:07:05.451786 25014 leader_election.cc:302] T 00000000000000000000000000000000 P 404a130a1eb34246a468c5d840b4ef0b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 404a130a1eb34246a468c5d840b4ef0b; no voters:
I20260809 02:07:05.452157 25014 leader_election.cc:288] T 00000000000000000000000000000000 P 404a130a1eb34246a468c5d840b4ef0b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:07:05.452297 25025 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 404a130a1eb34246a468c5d840b4ef0b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:07:05.452622 25025 raft_consensus.cc:686] T 00000000000000000000000000000000 P 404a130a1eb34246a468c5d840b4ef0b [term 1 LEADER]: Becoming Leader. State: Replica: 404a130a1eb34246a468c5d840b4ef0b, State: Running, Role: LEADER
I20260809 02:07:05.453083 25025 consensus_queue.cc:237] T 00000000000000000000000000000000 P 404a130a1eb34246a468c5d840b4ef0b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "404a130a1eb34246a468c5d840b4ef0b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40901 } }
I20260809 02:07:05.453687 25014 sys_catalog.cc:564] T 00000000000000000000000000000000 P 404a130a1eb34246a468c5d840b4ef0b [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:07:05.454591 25025 sys_catalog.cc:455] T 00000000000000000000000000000000 P 404a130a1eb34246a468c5d840b4ef0b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 404a130a1eb34246a468c5d840b4ef0b. Latest consensus state: current_term: 1 leader_uuid: "404a130a1eb34246a468c5d840b4ef0b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "404a130a1eb34246a468c5d840b4ef0b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40901 } } }
I20260809 02:07:05.454818 25025 sys_catalog.cc:458] T 00000000000000000000000000000000 P 404a130a1eb34246a468c5d840b4ef0b [sys.catalog]: This master's current role is: LEADER
I20260809 02:07:05.455052 25027 sys_catalog.cc:455] T 00000000000000000000000000000000 P 404a130a1eb34246a468c5d840b4ef0b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "404a130a1eb34246a468c5d840b4ef0b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "404a130a1eb34246a468c5d840b4ef0b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40901 } } }
I20260809 02:07:05.455153 25027 sys_catalog.cc:458] T 00000000000000000000000000000000 P 404a130a1eb34246a468c5d840b4ef0b [sys.catalog]: This master's current role is: LEADER
I20260809 02:07:05.455523 25017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:07:05.450028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.245:46825 (local address 127.0.0.1:40901)
0809 02:07:05.450199 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0809 02:07:05.450204 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:07:05.450343 (+ 139us) server_negotiation.cc:407] Connection header received
0809 02:07:05.450431 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:07:05.450434 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:07:05.450486 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:07:05.450578 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:07:05.450827 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:07:05.451635 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:07:05.452994 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:07:05.453332 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:07:05.453444 (+ 112us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:07:05.454579 (+ 1135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:07:05.454584 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:07:05.454587 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:07:05.454606 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:07:05.454642 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:07:05.454658 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:07:05.454740 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 02:07:05.455074 (+ 334us) server_negotiation.cc:299] Negotiation successful
0809 02:07:05.455122 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 02:07:05.455617 25034 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:07:05.459204 25034 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:07:05.465590 25034 catalog_manager.cc:1347] Generated new cluster ID: e619f39b3c224c2daa43f56885498e94
I20260809 02:07:05.465659 25034 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:07:05.485229 25034 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:07:05.486470 25034 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:07:05.502203 25034 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 404a130a1eb34246a468c5d840b4ef0b: Generated new TSK 0
I20260809 02:07:05.503062 25034 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:07:05.515731 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 24943
W20260809 02:07:05.524073 24927 connection.cc:535] client connection to 127.0.0.1:40901 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40901): Connection reset by peer (error 104)
2026-08-09T02:07:05Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1594 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786240941968412-1795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:07:05 dist-test-slave-ss4n krb5kdc[25047](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:07:05 dist-test-slave-ss4n krb5kdc[25047](info): set up 2 sockets
Aug 09 02:07:05 dist-test-slave-ss4n krb5kdc[25047](info): commencing operation
krb5kdc: starting...
W20260809 02:07:06.684716 1795 mini_kdc.cc:121] Time spent starting KDC: real 1.145s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786240941968412-1795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786240941968412-1795-0/krb5kdc/test-user.keytab.
Aug 09 02:07:06 dist-test-slave-ss4n krb5kdc[25047](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241226, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:07:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:07:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786240941968412-1795-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786240941968412-1795-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 02:07:06.829809 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786240941968412-1795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786240941968412-1795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786240941968412-1795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42379
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786240941968412-1795-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42379
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:07:06.948344 25060 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:07:06.948655 25060 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:07:06.948709 25060 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:07:06.952975 25060 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:07:06.953058 25060 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:07:06.953078 25060 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:07:06.953097 25060 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:07:06.953114 25060 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:07:06.958904 25060 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786240941968412-1795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786240941968412-1795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42379
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786240941968412-1795-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42379
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25060
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786240941968412-1795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:07:06.960139 25060 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:07:06.961242 25060 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 02:07:06.971375 25060 server_base.cc:1034] running on GCE node
W20260809 02:07:06.971316 25068 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:07:06.971304 25066 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:07:06.971307 25065 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:07:06.972218 25060 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:07:06.973464 25060 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:07:06.974737 25060 hybrid_clock.cc:648] HybridClock initialized: now 1786241226974712 us; error 39 us; skew 500 ppm
Aug 09 02:07:06 dist-test-slave-ss4n krb5kdc[25047](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241226, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:07:06.977949 25060 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 02:07:06.979030 25060 webserver.cc:460] Webserver started at http://127.0.0.1:46243/ using document root <none> and password file <none>
I20260809 02:07:06.979615 25060 fs_manager.cc:359] Metadata directory not provided
I20260809 02:07:06.979688 25060 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:07:06.979941 25060 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:07:06.982234 25060 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786240941968412-1795-0/minicluster-data/master-0/data/instance:
uuid: "497e332ed1cb4be2a73cf684fe8d4d75"
format_stamp: "Formatted at 2026-08-09 02:07:06 on dist-test-slave-ss4n"
server_key: "5f0424e70aff37633fb70a0df3dff2fe"
server_key_iv: "abab17488d1d1009ced8809065654b8f"
server_key_version: "encryptionkey@0"
I20260809 02:07:06.982846 25060 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786240941968412-1795-0/minicluster-data/master-0/wal/instance:
uuid: "497e332ed1cb4be2a73cf684fe8d4d75"
format_stamp: "Formatted at 2026-08-09 02:07:06 on dist-test-slave-ss4n"
server_key: "5f0424e70aff37633fb70a0df3dff2fe"
server_key_iv: "abab17488d1d1009ced8809065654b8f"
server_key_version: "encryptionkey@0"
I20260809 02:07:06.986984 25060 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 02:07:06.989743 25075 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:07:06.991257 25060 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 02:07:06.991442 25060 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786240941968412-1795-0/minicluster-data/master-0/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786240941968412-1795-0/minicluster-data/master-0/wal
uuid: "497e332ed1cb4be2a73cf684fe8d4d75"
format_stamp: "Formatted at 2026-08-09 02:07:06 on dist-test-slave-ss4n"
server_key: "5f0424e70aff37633fb70a0df3dff2fe"
server_key_iv: "abab17488d1d1009ced8809065654b8f"
server_key_version: "encryptionkey@0"
I20260809 02:07:06.991588 25060 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786240941968412-1795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786240941968412-1795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786240941968412-1795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:07:06.996981 25060 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:07:07.000871 25060 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:07:07.001111 25060 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:07:07.009390 25060 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42379
I20260809 02:07:07.009420 25132 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42379 every 8 connection(s)
I20260809 02:07:07.010555 25060 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
I20260809 02:07:07.013793 25133 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:07:07.018361 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 25060
I20260809 02:07:07.018520 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786240941968412-1795-0/minicluster-data/master-0/wal/instance
I20260809 02:07:07.018828 1795 external_mini_cluster.cc:1428] Setting key 752e0ecd20d51d49159d2027d9f5d8d4
I20260809 02:07:07.020102 25133 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 497e332ed1cb4be2a73cf684fe8d4d75: Bootstrap starting.
I20260809 02:07:07.022549 25133 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 497e332ed1cb4be2a73cf684fe8d4d75: Neither blocks nor log segments found. Creating new log.
I20260809 02:07:07.023519 25133 log.cc:824] T 00000000000000000000000000000000 P 497e332ed1cb4be2a73cf684fe8d4d75: Log is configured to *not* fsync() on all Append() calls
I20260809 02:07:07.025075 25136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:07:07.020520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.245:56515 (local address 127.0.0.1:42379)
0809 02:07:07.021016 (+ 496us) server_negotiation.cc:206] Beginning negotiation
0809 02:07:07.021032 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 02:07:07.021072 (+ 40us) server_negotiation.cc:407] Connection header received
0809 02:07:07.021871 (+ 799us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:07:07.021892 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:07:07.022228 (+ 336us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:07:07.022539 (+ 311us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:07:07.023542 (+ 1003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:07:07.023548 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:07:07.023561 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:07:07.023575 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:07:07.023740 (+ 165us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:07:07.023782 (+ 42us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:07:07.023890 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 02:07:07.024094 (+ 204us) server_negotiation.cc:299] Negotiation successful
0809 02:07:07.024262 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":312,"thread_start_us":149,"threads_started":1}
I20260809 02:07:07.025924 25133 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 497e332ed1cb4be2a73cf684fe8d4d75: No bootstrap required, opened a new log
I20260809 02:07:07.028330 1795 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 02:07:07.029963 25133 raft_consensus.cc:348] T 00000000000000000000000000000000 P 497e332ed1cb4be2a73cf684fe8d4d75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "497e332ed1cb4be2a73cf684fe8d4d75" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42379 } }
I20260809 02:07:07.030246 25133 raft_consensus.cc:374] T 00000000000000000000000000000000 P 497e332ed1cb4be2a73cf684fe8d4d75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:07:07.030339 25133 raft_consensus.cc:729] T 00000000000000000000000000000000 P 497e332ed1cb4be2a73cf684fe8d4d75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 497e332ed1cb4be2a73cf684fe8d4d75, State: Initialized, Role: FOLLOWER
I20260809 02:07:07.030879 25133 consensus_queue.cc:260] T 00000000000000000000000000000000 P 497e332ed1cb4be2a73cf684fe8d4d75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "497e332ed1cb4be2a73cf684fe8d4d75" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42379 } }
I20260809 02:07:07.031070 25133 raft_consensus.cc:388] T 00000000000000000000000000000000 P 497e332ed1cb4be2a73cf684fe8d4d75 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:07:07.031169 25133 raft_consensus.cc:482] T 00000000000000000000000000000000 P 497e332ed1cb4be2a73cf684fe8d4d75 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:07:07.031273 25133 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 497e332ed1cb4be2a73cf684fe8d4d75 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:07:07.032727 25136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:07:07.030555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.245:47269 (local address 127.0.0.1:42379)
0809 02:07:07.030744 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0809 02:07:07.030748 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:07:07.031016 (+ 268us) server_negotiation.cc:407] Connection header received
0809 02:07:07.031110 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:07:07.031113 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:07:07.031163 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:07:07.031275 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:07:07.032201 (+ 926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:07:07.032204 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:07:07.032207 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:07:07.032219 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:07:07.032250 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:07:07.032265 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:07:07.032367 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 02:07:07.032510 (+ 143us) server_negotiation.cc:299] Negotiation successful
0809 02:07:07.032560 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
I20260809 02:07:07.032686 25133 raft_consensus.cc:504] T 00000000000000000000000000000000 P 497e332ed1cb4be2a73cf684fe8d4d75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "497e332ed1cb4be2a73cf684fe8d4d75" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42379 } }
I20260809 02:07:07.033318 25133 leader_election.cc:302] T 00000000000000000000000000000000 P 497e332ed1cb4be2a73cf684fe8d4d75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 497e332ed1cb4be2a73cf684fe8d4d75; no voters:
I20260809 02:07:07.033748 25133 leader_election.cc:288] T 00000000000000000000000000000000 P 497e332ed1cb4be2a73cf684fe8d4d75 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:07:07.033825 25143 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 497e332ed1cb4be2a73cf684fe8d4d75 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:07:07.034116 25143 raft_consensus.cc:686] T 00000000000000000000000000000000 P 497e332ed1cb4be2a73cf684fe8d4d75 [term 1 LEADER]: Becoming Leader. State: Replica: 497e332ed1cb4be2a73cf684fe8d4d75, State: Running, Role: LEADER
I20260809 02:07:07.034493 25143 consensus_queue.cc:237] T 00000000000000000000000000000000 P 497e332ed1cb4be2a73cf684fe8d4d75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "497e332ed1cb4be2a73cf684fe8d4d75" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42379 } }
I20260809 02:07:07.035326 25133 sys_catalog.cc:564] T 00000000000000000000000000000000 P 497e332ed1cb4be2a73cf684fe8d4d75 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:07:07.035857 25143 sys_catalog.cc:455] T 00000000000000000000000000000000 P 497e332ed1cb4be2a73cf684fe8d4d75 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "497e332ed1cb4be2a73cf684fe8d4d75" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "497e332ed1cb4be2a73cf684fe8d4d75" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42379 } } }
I20260809 02:07:07.036072 25143 sys_catalog.cc:458] T 00000000000000000000000000000000 P 497e332ed1cb4be2a73cf684fe8d4d75 [sys.catalog]: This master's current role is: LEADER
I20260809 02:07:07.036386 25144 sys_catalog.cc:455] T 00000000000000000000000000000000 P 497e332ed1cb4be2a73cf684fe8d4d75 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 497e332ed1cb4be2a73cf684fe8d4d75. Latest consensus state: current_term: 1 leader_uuid: "497e332ed1cb4be2a73cf684fe8d4d75" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "497e332ed1cb4be2a73cf684fe8d4d75" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42379 } } }
I20260809 02:07:07.036541 25144 sys_catalog.cc:458] T 00000000000000000000000000000000 P 497e332ed1cb4be2a73cf684fe8d4d75 [sys.catalog]: This master's current role is: LEADER
I20260809 02:07:07.036633 25149 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:07:07.040223 25149 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:07:07.046931 25149 catalog_manager.cc:1347] Generated new cluster ID: 3828c123230445dbb718b7444870819b
I20260809 02:07:07.047037 25149 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:07:07.059542 25149 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:07:07.059983 25149 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:07:07.070341 25149 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 497e332ed1cb4be2a73cf684fe8d4d75: Generated new TSK 0
I20260809 02:07:07.071305 25149 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:07:07.082043 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 25060
2026-08-09T02:07:07Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1565 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786240941968412-1795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:07:07 dist-test-slave-ss4n krb5kdc[25166](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:07:07 dist-test-slave-ss4n krb5kdc[25166](info): set up 2 sockets
Aug 09 02:07:07 dist-test-slave-ss4n krb5kdc[25166](info): commencing operation
krb5kdc: starting...
W20260809 02:07:08.248003 1795 mini_kdc.cc:121] Time spent starting KDC: real 1.143s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786240941968412-1795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786240941968412-1795-0/krb5kdc/test-user.keytab.
Aug 09 02:07:08 dist-test-slave-ss4n krb5kdc[25166](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241228, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:07:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:07:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.192.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.192.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.192.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab.
Entry for principal kudu/127.1.192.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab.
I20260809 02:07:08.397883 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786240941968412-1795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786240941968412-1795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786240941968412-1795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.192.254:37713
--webserver_interface=127.1.192.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab
--principal=kudu/127.1.192.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:39245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.192.254:37713
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:07:08.516820 25179 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:07:08.517179 25179 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:07:08.517271 25179 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:07:08.521533 25179 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:07:08.521660 25179 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:07:08.521700 25179 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:07:08.521735 25179 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:07:08.521778 25179 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:07:08.527302 25179 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:39245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786240941968412-1795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786240941968412-1795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.192.254:37713
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab
--principal=kudu/127.1.192.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.192.254:37713
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.192.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25179
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786240941968412-1795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:07:08.528437 25179 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:07:08.529536 25179 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:07:08.538448 25188 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:07:08.538528 25185 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:07:08.538746 25179 server_base.cc:1034] running on GCE node
W20260809 02:07:08.538448 25186 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:07:08.539446 25179 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:07:08.540995 25179 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:07:08.542222 25179 hybrid_clock.cc:648] HybridClock initialized: now 1786241228542189 us; error 50 us; skew 500 ppm
Aug 09 02:07:08 dist-test-slave-ss4n krb5kdc[25166](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241228, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:07:08.545231 25179 init.cc:377] Logged in from keytab as kudu/127.1.192.254@KRBTEST.COM (short username kudu)
I20260809 02:07:08.546350 25179 webserver.cc:460] Webserver started at http://127.1.192.254:37803/ using document root <none> and password file <none>
I20260809 02:07:08.546907 25179 fs_manager.cc:359] Metadata directory not provided
I20260809 02:07:08.546969 25179 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:07:08.547142 25179 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:07:08.549232 25179 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786240941968412-1795-0/minicluster-data/master-0/data/instance:
uuid: "8cfea37950294d95bbfb09014375bf86"
format_stamp: "Formatted at 2026-08-09 02:07:08 on dist-test-slave-ss4n"
server_key: "d61e5b201abed2ad28ccd823f4f2704c"
server_key_iv: "efeab69fbd96d29c3ec10eddc307e827"
server_key_version: "encryptionkey@0"
I20260809 02:07:08.550289 25179 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786240941968412-1795-0/minicluster-data/master-0/wal/instance:
uuid: "8cfea37950294d95bbfb09014375bf86"
format_stamp: "Formatted at 2026-08-09 02:07:08 on dist-test-slave-ss4n"
server_key: "d61e5b201abed2ad28ccd823f4f2704c"
server_key_iv: "efeab69fbd96d29c3ec10eddc307e827"
server_key_version: "encryptionkey@0"
I20260809 02:07:08.555975 25179 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 02:07:08.558611 25195 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:07:08.560047 25179 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 02:07:08.560196 25179 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786240941968412-1795-0/minicluster-data/master-0/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786240941968412-1795-0/minicluster-data/master-0/wal
uuid: "8cfea37950294d95bbfb09014375bf86"
format_stamp: "Formatted at 2026-08-09 02:07:08 on dist-test-slave-ss4n"
server_key: "d61e5b201abed2ad28ccd823f4f2704c"
server_key_iv: "efeab69fbd96d29c3ec10eddc307e827"
server_key_version: "encryptionkey@0"
I20260809 02:07:08.560322 25179 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786240941968412-1795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786240941968412-1795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786240941968412-1795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:07:08.582314 25179 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:07:08.585709 25179 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:07:08.585952 25179 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:07:08.593343 25179 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.254:37713
I20260809 02:07:08.593372 25249 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.254:37713 every 8 connection(s)
I20260809 02:07:08.594384 25179 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
I20260809 02:07:08.596371 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 25179
I20260809 02:07:08.596540 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786240941968412-1795-0/minicluster-data/master-0/wal/instance
I20260809 02:07:08.596864 1795 external_mini_cluster.cc:1428] Setting key fc34710a3094f88702e6f209ded85a66
I20260809 02:07:08.597944 25250 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:07:08.603997 25250 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8cfea37950294d95bbfb09014375bf86: Bootstrap starting.
Aug 09 02:07:08 dist-test-slave-ss4n krb5kdc[25166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241228, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.192.254@KRBTEST.COM
I20260809 02:07:08.606735 25250 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8cfea37950294d95bbfb09014375bf86: Neither blocks nor log segments found. Creating new log.
I20260809 02:07:08.607649 25250 log.cc:824] T 00000000000000000000000000000000 P 8cfea37950294d95bbfb09014375bf86: Log is configured to *not* fsync() on all Append() calls
I20260809 02:07:08.610435 25250 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8cfea37950294d95bbfb09014375bf86: No bootstrap required, opened a new log
I20260809 02:07:08.612537 25253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:07:08.598473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39623 (local address 127.1.192.254:37713)
0809 02:07:08.598941 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0809 02:07:08.598961 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0809 02:07:08.598998 (+ 37us) server_negotiation.cc:407] Connection header received
0809 02:07:08.599634 (+ 636us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:07:08.599649 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:07:08.599971 (+ 322us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:07:08.600316 (+ 345us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:07:08.601191 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:07:08.602267 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:07:08.603159 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:07:08.603632 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:07:08.603806 (+ 174us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:07:08.606997 (+ 3191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:07:08.607003 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:07:08.607018 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:07:08.607048 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:07:08.609218 (+ 2170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:07:08.609901 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:07:08.609906 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:07:08.609911 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:07:08.610055 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:07:08.610456 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:07:08.610459 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:07:08.610461 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:07:08.610921 (+ 460us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:07:08.611167 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0809 02:07:08.611548 (+ 381us) server_negotiation.cc:299] Negotiation successful
0809 02:07:08.611789 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":326,"thread_start_us":151,"threads_started":1}
I20260809 02:07:08.614991 25250 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8cfea37950294d95bbfb09014375bf86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cfea37950294d95bbfb09014375bf86" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 37713 } }
I20260809 02:07:08.615300 25250 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8cfea37950294d95bbfb09014375bf86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:07:08.615397 25250 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8cfea37950294d95bbfb09014375bf86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8cfea37950294d95bbfb09014375bf86, State: Initialized, Role: FOLLOWER
I20260809 02:07:08.615742 1795 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 02:07:08.615970 25250 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8cfea37950294d95bbfb09014375bf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cfea37950294d95bbfb09014375bf86" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 37713 } }
I20260809 02:07:08.616153 25250 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8cfea37950294d95bbfb09014375bf86 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:07:08.616259 25250 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8cfea37950294d95bbfb09014375bf86 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:07:08.616375 25250 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8cfea37950294d95bbfb09014375bf86 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:07:08.617735 25250 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8cfea37950294d95bbfb09014375bf86 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cfea37950294d95bbfb09014375bf86" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 37713 } }
I20260809 02:07:08.618160 25250 leader_election.cc:302] T 00000000000000000000000000000000 P 8cfea37950294d95bbfb09014375bf86 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8cfea37950294d95bbfb09014375bf86; no voters:
I20260809 02:07:08.618604 25250 leader_election.cc:288] T 00000000000000000000000000000000 P 8cfea37950294d95bbfb09014375bf86 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:07:08.618722 25260 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8cfea37950294d95bbfb09014375bf86 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:07:08.618997 25260 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8cfea37950294d95bbfb09014375bf86 [term 1 LEADER]: Becoming Leader. State: Replica: 8cfea37950294d95bbfb09014375bf86, State: Running, Role: LEADER
I20260809 02:07:08.619346 25260 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8cfea37950294d95bbfb09014375bf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cfea37950294d95bbfb09014375bf86" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 37713 } }
I20260809 02:07:08.621089 25261 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8cfea37950294d95bbfb09014375bf86 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8cfea37950294d95bbfb09014375bf86. Latest consensus state: current_term: 1 leader_uuid: "8cfea37950294d95bbfb09014375bf86" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cfea37950294d95bbfb09014375bf86" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 37713 } } }
I20260809 02:07:08.621192 25250 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8cfea37950294d95bbfb09014375bf86 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:07:08.621255 25261 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8cfea37950294d95bbfb09014375bf86 [sys.catalog]: This master's current role is: LEADER
I20260809 02:07:08.621701 25264 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:07:08.621682 25262 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8cfea37950294d95bbfb09014375bf86 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8cfea37950294d95bbfb09014375bf86" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8cfea37950294d95bbfb09014375bf86" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 37713 } } }
I20260809 02:07:08.621814 25262 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8cfea37950294d95bbfb09014375bf86 [sys.catalog]: This master's current role is: LEADER
I20260809 02:07:08.625337 25264 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:07:08.628161 25253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:07:08.618134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50617 (local address 127.1.192.254:37713)
0809 02:07:08.618255 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0809 02:07:08.618259 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:07:08.618604 (+ 345us) server_negotiation.cc:407] Connection header received
0809 02:07:08.618712 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:07:08.618717 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:07:08.618769 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:07:08.618864 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:07:08.619773 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:07:08.620531 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:07:08.621405 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:07:08.621804 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:07:08.621904 (+ 100us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:07:08.623866 (+ 1962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:07:08.623870 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:07:08.623872 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:07:08.623898 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:07:08.625820 (+ 1922us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:07:08.626391 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:07:08.626395 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:07:08.626396 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:07:08.626482 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:07:08.626896 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:07:08.626899 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:07:08.626901 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:07:08.627109 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:07:08.627250 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0809 02:07:08.627915 (+ 665us) server_negotiation.cc:299] Negotiation successful
0809 02:07:08.628009 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 02:07:08.632391 25264 catalog_manager.cc:1347] Generated new cluster ID: 6347315568674d99a401fe67b4d7bc6e
I20260809 02:07:08.632479 25264 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:07:08.648854 25264 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:07:08.649972 25264 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:07:08.655684 25264 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8cfea37950294d95bbfb09014375bf86: Generated new TSK 0
I20260809 02:07:08.656608 25264 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:07:08.665825 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 25179
W20260809 02:07:08.674130 25163 connection.cc:535] client connection to 127.1.192.254:37713 recv error: Network error: failed to read from TLS socket (remote: 127.1.192.254:37713): Connection reset by peer (error 104)
2026-08-09T02:07:08Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1584 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786240941968412-1795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:07:08 dist-test-slave-ss4n krb5kdc[25283](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:07:08 dist-test-slave-ss4n krb5kdc[25283](info): set up 2 sockets
Aug 09 02:07:08 dist-test-slave-ss4n krb5kdc[25283](info): commencing operation
krb5kdc: starting...
W20260809 02:07:09.853644 1795 mini_kdc.cc:121] Time spent starting KDC: real 1.164s user 0.009s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786240941968412-1795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786240941968412-1795-0/krb5kdc/test-user.keytab.
Aug 09 02:07:09 dist-test-slave-ss4n krb5kdc[25283](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241229, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:07:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:07:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.192.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.192.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.192.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab.
Entry for principal kudu/127.1.192.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab.
I20260809 02:07:09.998196 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786240941968412-1795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786240941968412-1795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786240941968412-1795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.192.254:35771
--webserver_interface=127.1.192.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab
--principal=kudu/127.1.192.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:35109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.192.254:35771
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:07:10.117224 25297 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:07:10.117615 25297 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:07:10.117669 25297 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:07:10.121861 25297 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:07:10.121937 25297 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:07:10.121956 25297 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:07:10.121974 25297 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:07:10.122006 25297 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:07:10.127552 25297 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:35109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786240941968412-1795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786240941968412-1795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.192.254:35771
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab
--principal=kudu/127.1.192.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.192.254:35771
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.192.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25297
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786240941968412-1795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:07:10.128696 25297 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:07:10.129878 25297 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:07:10.139393 25302 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:07:10.139389 25305 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:07:10.139394 25303 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:07:10.139673 25297 server_base.cc:1034] running on GCE node
I20260809 02:07:10.140326 25297 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:07:10.141613 25297 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:07:10.142836 25297 hybrid_clock.cc:648] HybridClock initialized: now 1786241230142808 us; error 53 us; skew 500 ppm
Aug 09 02:07:10 dist-test-slave-ss4n krb5kdc[25283](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241230, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:07:10.146394 25297 init.cc:377] Logged in from keytab as kudu/127.1.192.254@KRBTEST.COM (short username kudu)
I20260809 02:07:10.147819 25297 webserver.cc:460] Webserver started at http://127.1.192.254:46859/ using document root <none> and password file <none>
I20260809 02:07:10.148505 25297 fs_manager.cc:359] Metadata directory not provided
I20260809 02:07:10.148619 25297 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:07:10.148891 25297 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:07:10.151465 25297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786240941968412-1795-0/minicluster-data/master-0/data/instance:
uuid: "29c8ada3d5a54210b4d4be2875bddd0d"
format_stamp: "Formatted at 2026-08-09 02:07:10 on dist-test-slave-ss4n"
server_key: "a51b6926f2d51b2f1d25d5d6b98401b2"
server_key_iv: "82490a08af31d34b9b8c3ba48c2adbfa"
server_key_version: "encryptionkey@0"
I20260809 02:07:10.152245 25297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786240941968412-1795-0/minicluster-data/master-0/wal/instance:
uuid: "29c8ada3d5a54210b4d4be2875bddd0d"
format_stamp: "Formatted at 2026-08-09 02:07:10 on dist-test-slave-ss4n"
server_key: "a51b6926f2d51b2f1d25d5d6b98401b2"
server_key_iv: "82490a08af31d34b9b8c3ba48c2adbfa"
server_key_version: "encryptionkey@0"
I20260809 02:07:10.156983 25297 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 02:07:10.160156 25313 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:07:10.161831 25297 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 02:07:10.162050 25297 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786240941968412-1795-0/minicluster-data/master-0/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786240941968412-1795-0/minicluster-data/master-0/wal
uuid: "29c8ada3d5a54210b4d4be2875bddd0d"
format_stamp: "Formatted at 2026-08-09 02:07:10 on dist-test-slave-ss4n"
server_key: "a51b6926f2d51b2f1d25d5d6b98401b2"
server_key_iv: "82490a08af31d34b9b8c3ba48c2adbfa"
server_key_version: "encryptionkey@0"
I20260809 02:07:10.162223 25297 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786240941968412-1795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786240941968412-1795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786240941968412-1795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:07:10.180564 25297 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:07:10.184576 25297 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:07:10.184824 25297 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:07:10.193305 25297 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.254:35771
I20260809 02:07:10.193349 25368 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.254:35771 every 8 connection(s)
I20260809 02:07:10.194438 25297 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
I20260809 02:07:10.196707 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 25297
I20260809 02:07:10.196877 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786240941968412-1795-0/minicluster-data/master-0/wal/instance
I20260809 02:07:10.197190 1795 external_mini_cluster.cc:1428] Setting key 8f31430cd8ff3105370ffffc93ae2b98
I20260809 02:07:10.197443 25369 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:07:10.204064 25369 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 29c8ada3d5a54210b4d4be2875bddd0d: Bootstrap starting.
I20260809 02:07:10.206493 25369 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 29c8ada3d5a54210b4d4be2875bddd0d: Neither blocks nor log segments found. Creating new log.
I20260809 02:07:10.207260 25369 log.cc:824] T 00000000000000000000000000000000 P 29c8ada3d5a54210b4d4be2875bddd0d: Log is configured to *not* fsync() on all Append() calls
I20260809 02:07:10.207417 25372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:07:10.198790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51211 (local address 127.1.192.254:35771)
0809 02:07:10.199280 (+ 490us) server_negotiation.cc:206] Beginning negotiation
0809 02:07:10.199296 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 02:07:10.199333 (+ 37us) server_negotiation.cc:407] Connection header received
0809 02:07:10.200275 (+ 942us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:07:10.200295 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:07:10.200661 (+ 366us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:07:10.201030 (+ 369us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:07:10.201455 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:07:10.202631 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:07:10.203632 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:07:10.204154 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:07:10.204334 (+ 180us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:07:10.205439 (+ 1105us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:07:10.205444 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:07:10.205457 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:07:10.205964 (+ 507us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:07:10.206034 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:07:10.206060 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:07:10.206215 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0809 02:07:10.206416 (+ 201us) server_negotiation.cc:299] Negotiation successful
0809 02:07:10.206574 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":179,"threads_started":1}
I20260809 02:07:10.209422 25369 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 29c8ada3d5a54210b4d4be2875bddd0d: No bootstrap required, opened a new log
I20260809 02:07:10.210698 1795 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 02:07:10.213438 25369 raft_consensus.cc:348] T 00000000000000000000000000000000 P 29c8ada3d5a54210b4d4be2875bddd0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29c8ada3d5a54210b4d4be2875bddd0d" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 35771 } }
I20260809 02:07:10.213797 25369 raft_consensus.cc:374] T 00000000000000000000000000000000 P 29c8ada3d5a54210b4d4be2875bddd0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:07:10.213881 25369 raft_consensus.cc:729] T 00000000000000000000000000000000 P 29c8ada3d5a54210b4d4be2875bddd0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29c8ada3d5a54210b4d4be2875bddd0d, State: Initialized, Role: FOLLOWER
I20260809 02:07:10.214381 25369 consensus_queue.cc:260] T 00000000000000000000000000000000 P 29c8ada3d5a54210b4d4be2875bddd0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29c8ada3d5a54210b4d4be2875bddd0d" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 35771 } }
I20260809 02:07:10.214541 25369 raft_consensus.cc:388] T 00000000000000000000000000000000 P 29c8ada3d5a54210b4d4be2875bddd0d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:07:10.214596 25369 raft_consensus.cc:482] T 00000000000000000000000000000000 P 29c8ada3d5a54210b4d4be2875bddd0d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:07:10.214690 25369 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 29c8ada3d5a54210b4d4be2875bddd0d [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:07:10.215927 25369 raft_consensus.cc:504] T 00000000000000000000000000000000 P 29c8ada3d5a54210b4d4be2875bddd0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29c8ada3d5a54210b4d4be2875bddd0d" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 35771 } }
I20260809 02:07:10.216325 25369 leader_election.cc:302] T 00000000000000000000000000000000 P 29c8ada3d5a54210b4d4be2875bddd0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 29c8ada3d5a54210b4d4be2875bddd0d; no voters:
I20260809 02:07:10.216708 25369 leader_election.cc:288] T 00000000000000000000000000000000 P 29c8ada3d5a54210b4d4be2875bddd0d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:07:10.216825 25379 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 29c8ada3d5a54210b4d4be2875bddd0d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:07:10.217093 25379 raft_consensus.cc:686] T 00000000000000000000000000000000 P 29c8ada3d5a54210b4d4be2875bddd0d [term 1 LEADER]: Becoming Leader. State: Replica: 29c8ada3d5a54210b4d4be2875bddd0d, State: Running, Role: LEADER
I20260809 02:07:10.217451 25379 consensus_queue.cc:237] T 00000000000000000000000000000000 P 29c8ada3d5a54210b4d4be2875bddd0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29c8ada3d5a54210b4d4be2875bddd0d" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 35771 } }
I20260809 02:07:10.217953 25372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:07:10.212903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41557 (local address 127.1.192.254:35771)
0809 02:07:10.213034 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0809 02:07:10.213039 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:07:10.213315 (+ 276us) server_negotiation.cc:407] Connection header received
0809 02:07:10.213391 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:07:10.213393 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:07:10.213434 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:07:10.213602 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:07:10.213921 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:07:10.214667 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:07:10.215843 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:07:10.216176 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:07:10.216264 (+ 88us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:07:10.217326 (+ 1062us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:07:10.217330 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:07:10.217332 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:07:10.217345 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:07:10.217374 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:07:10.217386 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:07:10.217444 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 02:07:10.217765 (+ 321us) server_negotiation.cc:299] Negotiation successful
0809 02:07:10.217811 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 02:07:10.218195 25369 sys_catalog.cc:564] T 00000000000000000000000000000000 P 29c8ada3d5a54210b4d4be2875bddd0d [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:07:10.219451 25381 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29c8ada3d5a54210b4d4be2875bddd0d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "29c8ada3d5a54210b4d4be2875bddd0d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29c8ada3d5a54210b4d4be2875bddd0d" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 35771 } } }
I20260809 02:07:10.219597 25381 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29c8ada3d5a54210b4d4be2875bddd0d [sys.catalog]: This master's current role is: LEADER
I20260809 02:07:10.219986 25382 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29c8ada3d5a54210b4d4be2875bddd0d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 29c8ada3d5a54210b4d4be2875bddd0d. Latest consensus state: current_term: 1 leader_uuid: "29c8ada3d5a54210b4d4be2875bddd0d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29c8ada3d5a54210b4d4be2875bddd0d" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 35771 } } }
I20260809 02:07:10.220120 25382 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29c8ada3d5a54210b4d4be2875bddd0d [sys.catalog]: This master's current role is: LEADER
W20260809 02:07:10.223824 25395 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 29c8ada3d5a54210b4d4be2875bddd0d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 02:07:10.223924 25395 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 02:07:10.224085 25390 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:07:10.225013 25390 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:07:10.231436 25390 catalog_manager.cc:1347] Generated new cluster ID: 8e9448fddadb4edfa9365808d3ec8c3c
I20260809 02:07:10.231637 25390 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:07:10.240566 25390 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:07:10.241734 25390 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:07:10.248239 25390 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 29c8ada3d5a54210b4d4be2875bddd0d: Generated new TSK 0
I20260809 02:07:10.249080 25390 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:07:10.258895 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 25297
W20260809 02:07:10.266710 25281 connection.cc:535] client connection to 127.1.192.254:35771 recv error: Network error: failed to read from TLS socket (remote: 127.1.192.254:35771): Connection reset by peer (error 104)
2026-08-09T02:07:10Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1592 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786240941968412-1795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:07:10 dist-test-slave-ss4n krb5kdc[25401](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:07:10 dist-test-slave-ss4n krb5kdc[25401](info): set up 2 sockets
Aug 09 02:07:10 dist-test-slave-ss4n krb5kdc[25401](info): commencing operation
krb5kdc: starting...
W20260809 02:07:11.425829 1795 mini_kdc.cc:121] Time spent starting KDC: real 1.144s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786240941968412-1795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786240941968412-1795-0/krb5kdc/test-user.keytab.
Aug 09 02:07:11 dist-test-slave-ss4n krb5kdc[25401](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241231, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:07:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:07:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.192.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.192.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.192.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab.
Entry for principal kudu/127.1.192.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab.
I20260809 02:07:11.572834 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786240941968412-1795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786240941968412-1795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786240941968412-1795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.192.254:38223
--webserver_interface=127.1.192.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab
--principal=kudu/127.1.192.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:42711
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.192.254:38223
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:07:11.695546 25414 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:07:11.695928 25414 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:07:11.696022 25414 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:07:11.700238 25414 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:07:11.700361 25414 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:07:11.700402 25414 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:07:11.700440 25414 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:07:11.700470 25414 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:07:11.706135 25414 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:42711
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786240941968412-1795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786240941968412-1795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.192.254:38223
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab
--principal=kudu/127.1.192.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.192.254:38223
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.192.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25414
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786240941968412-1795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:07:11.707355 25414 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:07:11.708473 25414 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:07:11.718506 25421 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:07:11.718637 25420 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:07:11.718677 25423 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:07:11.718844 25414 server_base.cc:1034] running on GCE node
I20260809 02:07:11.719560 25414 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:07:11.720930 25414 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:07:11.722141 25414 hybrid_clock.cc:648] HybridClock initialized: now 1786241231722109 us; error 47 us; skew 500 ppm
Aug 09 02:07:11 dist-test-slave-ss4n krb5kdc[25401](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241231, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:07:11.725442 25414 init.cc:377] Logged in from keytab as kudu/127.1.192.254@KRBTEST.COM (short username kudu)
I20260809 02:07:11.726709 25414 webserver.cc:460] Webserver started at http://127.1.192.254:38187/ using document root <none> and password file <none>
I20260809 02:07:11.727366 25414 fs_manager.cc:359] Metadata directory not provided
I20260809 02:07:11.727438 25414 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:07:11.727694 25414 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:07:11.730078 25414 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786240941968412-1795-0/minicluster-data/master-0/data/instance:
uuid: "ecb4e04e225e464cb6a473731de1afa2"
format_stamp: "Formatted at 2026-08-09 02:07:11 on dist-test-slave-ss4n"
server_key: "0f54dbf089da9e06386343f5518de20b"
server_key_iv: "2e931be20f6fcfcdc635858eb7fac568"
server_key_version: "encryptionkey@0"
I20260809 02:07:11.730705 25414 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786240941968412-1795-0/minicluster-data/master-0/wal/instance:
uuid: "ecb4e04e225e464cb6a473731de1afa2"
format_stamp: "Formatted at 2026-08-09 02:07:11 on dist-test-slave-ss4n"
server_key: "0f54dbf089da9e06386343f5518de20b"
server_key_iv: "2e931be20f6fcfcdc635858eb7fac568"
server_key_version: "encryptionkey@0"
I20260809 02:07:11.734920 25414 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 02:07:11.737676 25430 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:07:11.739092 25414 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 02:07:11.739261 25414 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786240941968412-1795-0/minicluster-data/master-0/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786240941968412-1795-0/minicluster-data/master-0/wal
uuid: "ecb4e04e225e464cb6a473731de1afa2"
format_stamp: "Formatted at 2026-08-09 02:07:11 on dist-test-slave-ss4n"
server_key: "0f54dbf089da9e06386343f5518de20b"
server_key_iv: "2e931be20f6fcfcdc635858eb7fac568"
server_key_version: "encryptionkey@0"
I20260809 02:07:11.739383 25414 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786240941968412-1795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786240941968412-1795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786240941968412-1795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:07:11.744495 25414 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:07:11.748147 25414 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:07:11.748368 25414 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:07:11.756356 25414 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.254:38223
I20260809 02:07:11.756385 25487 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.254:38223 every 8 connection(s)
I20260809 02:07:11.757381 25414 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
I20260809 02:07:11.760368 25488 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:07:11.761688 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 25414
I20260809 02:07:11.761842 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786240941968412-1795-0/minicluster-data/master-0/wal/instance
I20260809 02:07:11.762156 1795 external_mini_cluster.cc:1428] Setting key 257ef1daa3f0b42c124969df7ba7c821
I20260809 02:07:11.767366 25488 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ecb4e04e225e464cb6a473731de1afa2: Bootstrap starting.
I20260809 02:07:11.767865 25491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:07:11.763779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54295 (local address 127.1.192.254:38223)
0809 02:07:11.764271 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0809 02:07:11.764286 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 02:07:11.764323 (+ 37us) server_negotiation.cc:407] Connection header received
0809 02:07:11.765002 (+ 679us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:07:11.765022 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:07:11.765343 (+ 321us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:07:11.765749 (+ 406us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:07:11.766604 (+ 855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:07:11.766608 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:07:11.766620 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:07:11.766630 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:07:11.766687 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:07:11.766707 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:07:11.766870 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0809 02:07:11.766941 (+ 71us) server_negotiation.cc:299] Negotiation successful
0809 02:07:11.767108 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":156,"threads_started":1}
I20260809 02:07:11.770110 25488 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ecb4e04e225e464cb6a473731de1afa2: Neither blocks nor log segments found. Creating new log.
I20260809 02:07:11.771068 25488 log.cc:824] T 00000000000000000000000000000000 P ecb4e04e225e464cb6a473731de1afa2: Log is configured to *not* fsync() on all Append() calls
I20260809 02:07:11.771270 1795 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 02:07:11.773279 25488 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ecb4e04e225e464cb6a473731de1afa2: No bootstrap required, opened a new log
I20260809 02:07:11.775770 25491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:07:11.773808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49295 (local address 127.1.192.254:38223)
0809 02:07:11.773965 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0809 02:07:11.773970 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:07:11.774175 (+ 205us) server_negotiation.cc:407] Connection header received
0809 02:07:11.774287 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:07:11.774290 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:07:11.774323 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:07:11.774404 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:07:11.775312 (+ 908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:07:11.775315 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:07:11.775317 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:07:11.775328 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:07:11.775356 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:07:11.775366 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:07:11.775451 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 02:07:11.775557 (+ 106us) server_negotiation.cc:299] Negotiation successful
0809 02:07:11.775591 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 02:07:11.778152 25488 raft_consensus.cc:348] T 00000000000000000000000000000000 P ecb4e04e225e464cb6a473731de1afa2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecb4e04e225e464cb6a473731de1afa2" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 38223 } }
I20260809 02:07:11.778424 25488 raft_consensus.cc:374] T 00000000000000000000000000000000 P ecb4e04e225e464cb6a473731de1afa2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:07:11.778486 25488 raft_consensus.cc:729] T 00000000000000000000000000000000 P ecb4e04e225e464cb6a473731de1afa2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ecb4e04e225e464cb6a473731de1afa2, State: Initialized, Role: FOLLOWER
I20260809 02:07:11.778962 25488 consensus_queue.cc:260] T 00000000000000000000000000000000 P ecb4e04e225e464cb6a473731de1afa2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecb4e04e225e464cb6a473731de1afa2" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 38223 } }
I20260809 02:07:11.779147 25488 raft_consensus.cc:388] T 00000000000000000000000000000000 P ecb4e04e225e464cb6a473731de1afa2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:07:11.779242 25488 raft_consensus.cc:482] T 00000000000000000000000000000000 P ecb4e04e225e464cb6a473731de1afa2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:07:11.779348 25488 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ecb4e04e225e464cb6a473731de1afa2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:07:11.780426 25488 raft_consensus.cc:504] T 00000000000000000000000000000000 P ecb4e04e225e464cb6a473731de1afa2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecb4e04e225e464cb6a473731de1afa2" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 38223 } }
I20260809 02:07:11.780804 25488 leader_election.cc:302] T 00000000000000000000000000000000 P ecb4e04e225e464cb6a473731de1afa2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ecb4e04e225e464cb6a473731de1afa2; no voters:
I20260809 02:07:11.781168 25488 leader_election.cc:288] T 00000000000000000000000000000000 P ecb4e04e225e464cb6a473731de1afa2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:07:11.781376 25498 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ecb4e04e225e464cb6a473731de1afa2 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:07:11.781769 25498 raft_consensus.cc:686] T 00000000000000000000000000000000 P ecb4e04e225e464cb6a473731de1afa2 [term 1 LEADER]: Becoming Leader. State: Replica: ecb4e04e225e464cb6a473731de1afa2, State: Running, Role: LEADER
I20260809 02:07:11.782135 25498 consensus_queue.cc:237] T 00000000000000000000000000000000 P ecb4e04e225e464cb6a473731de1afa2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecb4e04e225e464cb6a473731de1afa2" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 38223 } }
I20260809 02:07:11.782701 25488 sys_catalog.cc:564] T 00000000000000000000000000000000 P ecb4e04e225e464cb6a473731de1afa2 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:07:11.784066 25500 sys_catalog.cc:455] T 00000000000000000000000000000000 P ecb4e04e225e464cb6a473731de1afa2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ecb4e04e225e464cb6a473731de1afa2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecb4e04e225e464cb6a473731de1afa2" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 38223 } } }
I20260809 02:07:11.784206 25500 sys_catalog.cc:458] T 00000000000000000000000000000000 P ecb4e04e225e464cb6a473731de1afa2 [sys.catalog]: This master's current role is: LEADER
I20260809 02:07:11.784533 25501 sys_catalog.cc:455] T 00000000000000000000000000000000 P ecb4e04e225e464cb6a473731de1afa2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ecb4e04e225e464cb6a473731de1afa2. Latest consensus state: current_term: 1 leader_uuid: "ecb4e04e225e464cb6a473731de1afa2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecb4e04e225e464cb6a473731de1afa2" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 38223 } } }
I20260809 02:07:11.784612 25501 sys_catalog.cc:458] T 00000000000000000000000000000000 P ecb4e04e225e464cb6a473731de1afa2 [sys.catalog]: This master's current role is: LEADER
I20260809 02:07:11.784986 25509 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:07:11.788152 25509 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:07:11.793906 25509 catalog_manager.cc:1347] Generated new cluster ID: ff0c8f3b09ba49bea3534ffd35133a4a
I20260809 02:07:11.793987 25509 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:07:11.815949 25509 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:07:11.816342 25509 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:07:11.822960 25509 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ecb4e04e225e464cb6a473731de1afa2: Generated new TSK 0
I20260809 02:07:11.823750 25509 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:07:11.828967 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 25414
2026-08-09T02:07:11Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1570 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786240941968412-1795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:07:11 dist-test-slave-ss4n krb5kdc[25521](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:07:11 dist-test-slave-ss4n krb5kdc[25521](info): set up 2 sockets
Aug 09 02:07:11 dist-test-slave-ss4n krb5kdc[25521](info): commencing operation
krb5kdc: starting...
W20260809 02:07:12.950174 1795 mini_kdc.cc:121] Time spent starting KDC: real 1.098s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786240941968412-1795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786240941968412-1795-0/krb5kdc/test-user.keytab.
Aug 09 02:07:13 dist-test-slave-ss4n krb5kdc[25521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241233, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:07:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:07:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.192.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.192.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.192.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab.
Entry for principal kudu/127.1.192.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab.
I20260809 02:07:13.094316 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786240941968412-1795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786240941968412-1795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786240941968412-1795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.192.254:41035
--webserver_interface=127.1.192.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab
--principal=kudu/127.1.192.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:41729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.192.254:41035
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:07:13.214486 25535 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:07:13.214843 25535 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:07:13.214942 25535 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:07:13.219355 25535 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:07:13.219462 25535 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:07:13.219497 25535 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:07:13.219523 25535 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:07:13.219544 25535 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:07:13.225172 25535 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:41729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786240941968412-1795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786240941968412-1795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.192.254:41035
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab
--principal=kudu/127.1.192.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.192.254:41035
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.192.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25535
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786240941968412-1795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:07:13.226377 25535 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:07:13.227450 25535 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:07:13.236521 25540 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:07:13.236567 25542 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:07:13.236595 25544 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:07:13.236742 25535 server_base.cc:1034] running on GCE node
I20260809 02:07:13.237432 25535 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:07:13.238718 25535 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:07:13.239914 25535 hybrid_clock.cc:648] HybridClock initialized: now 1786241233239887 us; error 41 us; skew 500 ppm
Aug 09 02:07:13 dist-test-slave-ss4n krb5kdc[25521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241233, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:07:13.243005 25535 init.cc:377] Logged in from keytab as kudu/127.1.192.254@KRBTEST.COM (short username kudu)
I20260809 02:07:13.243963 25535 webserver.cc:460] Webserver started at http://127.1.192.254:40741/ using document root <none> and password file <none>
I20260809 02:07:13.244510 25535 fs_manager.cc:359] Metadata directory not provided
I20260809 02:07:13.244572 25535 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:07:13.244741 25535 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:07:13.246882 25535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786240941968412-1795-0/minicluster-data/master-0/data/instance:
uuid: "184574863dc14ff486ab0799e324c45a"
format_stamp: "Formatted at 2026-08-09 02:07:13 on dist-test-slave-ss4n"
server_key: "9493a0ea87d94caa0b42a9b0cf490f0e"
server_key_iv: "38262a3b5a6aed2e9766ef4bea39137c"
server_key_version: "encryptionkey@0"
I20260809 02:07:13.247413 25535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786240941968412-1795-0/minicluster-data/master-0/wal/instance:
uuid: "184574863dc14ff486ab0799e324c45a"
format_stamp: "Formatted at 2026-08-09 02:07:13 on dist-test-slave-ss4n"
server_key: "9493a0ea87d94caa0b42a9b0cf490f0e"
server_key_iv: "38262a3b5a6aed2e9766ef4bea39137c"
server_key_version: "encryptionkey@0"
I20260809 02:07:13.251328 25535 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260809 02:07:13.254190 25551 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:07:13.255720 25535 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 02:07:13.255908 25535 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786240941968412-1795-0/minicluster-data/master-0/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786240941968412-1795-0/minicluster-data/master-0/wal
uuid: "184574863dc14ff486ab0799e324c45a"
format_stamp: "Formatted at 2026-08-09 02:07:13 on dist-test-slave-ss4n"
server_key: "9493a0ea87d94caa0b42a9b0cf490f0e"
server_key_iv: "38262a3b5a6aed2e9766ef4bea39137c"
server_key_version: "encryptionkey@0"
I20260809 02:07:13.256047 25535 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786240941968412-1795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786240941968412-1795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786240941968412-1795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:07:13.268559 25535 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:07:13.272208 25535 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:07:13.272451 25535 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:07:13.280995 25535 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.254:41035
I20260809 02:07:13.281029 25606 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.254:41035 every 8 connection(s)
I20260809 02:07:13.282218 25535 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
I20260809 02:07:13.282582 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 25535
I20260809 02:07:13.282752 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786240941968412-1795-0/minicluster-data/master-0/wal/instance
I20260809 02:07:13.283118 1795 external_mini_cluster.cc:1428] Setting key beb98ac0adf366802168839ae5632524
I20260809 02:07:13.285691 25607 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:07:13.293323 25607 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 184574863dc14ff486ab0799e324c45a: Bootstrap starting.
Aug 09 02:07:13 dist-test-slave-ss4n krb5kdc[25521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241233, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.192.254@KRBTEST.COM
I20260809 02:07:13.295899 25607 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 184574863dc14ff486ab0799e324c45a: Neither blocks nor log segments found. Creating new log.
I20260809 02:07:13.296746 25607 log.cc:824] T 00000000000000000000000000000000 P 184574863dc14ff486ab0799e324c45a: Log is configured to *not* fsync() on all Append() calls
I20260809 02:07:13.299048 25607 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 184574863dc14ff486ab0799e324c45a: No bootstrap required, opened a new log
I20260809 02:07:13.300307 25611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:07:13.284700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33749 (local address 127.1.192.254:41035)
0809 02:07:13.285124 (+ 424us) server_negotiation.cc:206] Beginning negotiation
0809 02:07:13.285149 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0809 02:07:13.285186 (+ 37us) server_negotiation.cc:407] Connection header received
0809 02:07:13.286041 (+ 855us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:07:13.286067 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:07:13.286491 (+ 424us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:07:13.286820 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:07:13.287928 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:07:13.288944 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:07:13.290352 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:07:13.291010 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:07:13.291184 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:07:13.294616 (+ 3432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:07:13.294622 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:07:13.294637 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:07:13.294672 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:07:13.296902 (+ 2230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:07:13.297514 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:07:13.297524 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:07:13.297532 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:07:13.297678 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:07:13.298046 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:07:13.298050 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:07:13.298052 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:07:13.298524 (+ 472us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:07:13.298767 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0809 02:07:13.299103 (+ 336us) server_negotiation.cc:299] Negotiation successful
0809 02:07:13.299451 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":147,"threads_started":1}
I20260809 02:07:13.302848 25607 raft_consensus.cc:348] T 00000000000000000000000000000000 P 184574863dc14ff486ab0799e324c45a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "184574863dc14ff486ab0799e324c45a" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 41035 } }
I20260809 02:07:13.303164 25607 raft_consensus.cc:374] T 00000000000000000000000000000000 P 184574863dc14ff486ab0799e324c45a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:07:13.303278 25607 raft_consensus.cc:729] T 00000000000000000000000000000000 P 184574863dc14ff486ab0799e324c45a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 184574863dc14ff486ab0799e324c45a, State: Initialized, Role: FOLLOWER
I20260809 02:07:13.303833 25607 consensus_queue.cc:260] T 00000000000000000000000000000000 P 184574863dc14ff486ab0799e324c45a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "184574863dc14ff486ab0799e324c45a" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 41035 } }
I20260809 02:07:13.304008 25607 raft_consensus.cc:388] T 00000000000000000000000000000000 P 184574863dc14ff486ab0799e324c45a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:07:13.304092 25607 raft_consensus.cc:482] T 00000000000000000000000000000000 P 184574863dc14ff486ab0799e324c45a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:07:13.304121 1795 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 02:07:13.304239 25607 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 184574863dc14ff486ab0799e324c45a [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:07:13.305361 25607 raft_consensus.cc:504] T 00000000000000000000000000000000 P 184574863dc14ff486ab0799e324c45a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "184574863dc14ff486ab0799e324c45a" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 41035 } }
I20260809 02:07:13.305871 25607 leader_election.cc:302] T 00000000000000000000000000000000 P 184574863dc14ff486ab0799e324c45a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 184574863dc14ff486ab0799e324c45a; no voters:
I20260809 02:07:13.306280 25607 leader_election.cc:288] T 00000000000000000000000000000000 P 184574863dc14ff486ab0799e324c45a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:07:13.306439 25617 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 184574863dc14ff486ab0799e324c45a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:07:13.306767 25617 raft_consensus.cc:686] T 00000000000000000000000000000000 P 184574863dc14ff486ab0799e324c45a [term 1 LEADER]: Becoming Leader. State: Replica: 184574863dc14ff486ab0799e324c45a, State: Running, Role: LEADER
I20260809 02:07:13.307217 25617 consensus_queue.cc:237] T 00000000000000000000000000000000 P 184574863dc14ff486ab0799e324c45a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "184574863dc14ff486ab0799e324c45a" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 41035 } }
I20260809 02:07:13.307835 25607 sys_catalog.cc:564] T 00000000000000000000000000000000 P 184574863dc14ff486ab0799e324c45a [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:07:13.308748 25617 sys_catalog.cc:455] T 00000000000000000000000000000000 P 184574863dc14ff486ab0799e324c45a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "184574863dc14ff486ab0799e324c45a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "184574863dc14ff486ab0799e324c45a" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 41035 } } }
I20260809 02:07:13.308969 25617 sys_catalog.cc:458] T 00000000000000000000000000000000 P 184574863dc14ff486ab0799e324c45a [sys.catalog]: This master's current role is: LEADER
I20260809 02:07:13.309252 25617 sys_catalog.cc:455] T 00000000000000000000000000000000 P 184574863dc14ff486ab0799e324c45a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 184574863dc14ff486ab0799e324c45a. Latest consensus state: current_term: 1 leader_uuid: "184574863dc14ff486ab0799e324c45a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "184574863dc14ff486ab0799e324c45a" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 41035 } } }
I20260809 02:07:13.309422 25617 sys_catalog.cc:458] T 00000000000000000000000000000000 P 184574863dc14ff486ab0799e324c45a [sys.catalog]: This master's current role is: LEADER
I20260809 02:07:13.309814 25627 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:07:13.313588 25627 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:07:13.317911 25611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:07:13.306557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49813 (local address 127.1.192.254:41035)
0809 02:07:13.306705 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0809 02:07:13.306712 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 02:07:13.307087 (+ 375us) server_negotiation.cc:407] Connection header received
0809 02:07:13.307213 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:07:13.307216 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:07:13.307273 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:07:13.307394 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:07:13.308441 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:07:13.309383 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:07:13.311014 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:07:13.311492 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:07:13.311602 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:07:13.313755 (+ 2153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:07:13.313761 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:07:13.313767 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:07:13.313805 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:07:13.315902 (+ 2097us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:07:13.316437 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:07:13.316441 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:07:13.316443 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:07:13.316546 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:07:13.316877 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:07:13.316882 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:07:13.316884 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:07:13.317109 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:07:13.317260 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0809 02:07:13.317586 (+ 326us) server_negotiation.cc:299] Negotiation successful
0809 02:07:13.317730 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 02:07:13.320737 25627 catalog_manager.cc:1347] Generated new cluster ID: ccedc79b42804b02a184a81994f29530
I20260809 02:07:13.320827 25627 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:07:13.328357 25627 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:07:13.329825 25627 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:07:13.343032 25627 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 184574863dc14ff486ab0799e324c45a: Generated new TSK 0
I20260809 02:07:13.344156 25627 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:07:13.348528 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 25535
2026-08-09T02:07:13Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1519 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786240941968412-1795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:07:13 dist-test-slave-ss4n krb5kdc[25640](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:07:13 dist-test-slave-ss4n krb5kdc[25640](info): set up 2 sockets
Aug 09 02:07:13 dist-test-slave-ss4n krb5kdc[25640](info): commencing operation
krb5kdc: starting...
W20260809 02:07:14.499940 1795 mini_kdc.cc:121] Time spent starting KDC: real 1.128s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786240941968412-1795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786240941968412-1795-0/krb5kdc/test-user.keytab.
Aug 09 02:07:14 dist-test-slave-ss4n krb5kdc[25640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241234, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:07:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:07:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.192.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.192.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.192.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab.
Entry for principal kudu/127.1.192.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab.
I20260809 02:07:14.645407 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786240941968412-1795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786240941968412-1795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786240941968412-1795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.192.254:37037
--webserver_interface=127.1.192.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab
--principal=kudu/127.1.192.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:33465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.192.254:37037
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:07:14.768217 25654 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:07:14.768581 25654 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:07:14.768671 25654 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:07:14.772961 25654 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:07:14.773037 25654 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:07:14.773058 25654 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:07:14.773075 25654 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:07:14.773123 25654 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:07:14.778995 25654 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:33465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786240941968412-1795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786240941968412-1795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.192.254:37037
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab
--principal=kudu/127.1.192.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.192.254:37037
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.192.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25654
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786240941968412-1795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:07:14.780277 25654 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:07:14.781354 25654 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:07:14.790690 25660 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:07:14.790773 25659 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:07:14.790766 25654 server_base.cc:1034] running on GCE node
W20260809 02:07:14.790680 25663 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:07:14.791579 25654 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:07:14.792976 25654 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:07:14.794205 25654 hybrid_clock.cc:648] HybridClock initialized: now 1786241234794194 us; error 34 us; skew 500 ppm
Aug 09 02:07:14 dist-test-slave-ss4n krb5kdc[25640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241234, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:07:14.797717 25654 init.cc:377] Logged in from keytab as kudu/127.1.192.254@KRBTEST.COM (short username kudu)
I20260809 02:07:14.799052 25654 webserver.cc:460] Webserver started at http://127.1.192.254:33187/ using document root <none> and password file <none>
I20260809 02:07:14.799605 25654 fs_manager.cc:359] Metadata directory not provided
I20260809 02:07:14.799665 25654 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:07:14.799830 25654 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:07:14.801983 25654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786240941968412-1795-0/minicluster-data/master-0/data/instance:
uuid: "20ab33241766413f961c013481231ec0"
format_stamp: "Formatted at 2026-08-09 02:07:14 on dist-test-slave-ss4n"
server_key: "26cd2e6df55e7f6847fb0dc6a0e461cf"
server_key_iv: "3de98ec0694401425c3b2f43aeec3993"
server_key_version: "encryptionkey@0"
I20260809 02:07:14.802574 25654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786240941968412-1795-0/minicluster-data/master-0/wal/instance:
uuid: "20ab33241766413f961c013481231ec0"
format_stamp: "Formatted at 2026-08-09 02:07:14 on dist-test-slave-ss4n"
server_key: "26cd2e6df55e7f6847fb0dc6a0e461cf"
server_key_iv: "3de98ec0694401425c3b2f43aeec3993"
server_key_version: "encryptionkey@0"
I20260809 02:07:14.806785 25654 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260809 02:07:14.809574 25671 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:07:14.811074 25654 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 02:07:14.811201 25654 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786240941968412-1795-0/minicluster-data/master-0/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786240941968412-1795-0/minicluster-data/master-0/wal
uuid: "20ab33241766413f961c013481231ec0"
format_stamp: "Formatted at 2026-08-09 02:07:14 on dist-test-slave-ss4n"
server_key: "26cd2e6df55e7f6847fb0dc6a0e461cf"
server_key_iv: "3de98ec0694401425c3b2f43aeec3993"
server_key_version: "encryptionkey@0"
I20260809 02:07:14.811461 25654 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786240941968412-1795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786240941968412-1795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786240941968412-1795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:07:14.824730 25654 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:07:14.828293 25654 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:07:14.828554 25654 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:07:14.836228 25654 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.254:37037
I20260809 02:07:14.836256 25724 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.254:37037 every 8 connection(s)
I20260809 02:07:14.837248 25654 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
I20260809 02:07:14.840211 25725 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:07:14.843758 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 25654
I20260809 02:07:14.843881 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786240941968412-1795-0/minicluster-data/master-0/wal/instance
I20260809 02:07:14.844202 1795 external_mini_cluster.cc:1428] Setting key 0ce70447df7455426dd127ec8ace4be5
I20260809 02:07:14.846910 25725 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 20ab33241766413f961c013481231ec0: Bootstrap starting.
I20260809 02:07:14.849298 25725 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 20ab33241766413f961c013481231ec0: Neither blocks nor log segments found. Creating new log.
I20260809 02:07:14.850207 25725 log.cc:824] T 00000000000000000000000000000000 P 20ab33241766413f961c013481231ec0: Log is configured to *not* fsync() on all Append() calls
I20260809 02:07:14.852430 25725 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 20ab33241766413f961c013481231ec0: No bootstrap required, opened a new log
I20260809 02:07:14.854683 25728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:07:14.845786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57491 (local address 127.1.192.254:37037)
0809 02:07:14.846319 (+ 533us) server_negotiation.cc:206] Beginning negotiation
0809 02:07:14.846345 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0809 02:07:14.846388 (+ 43us) server_negotiation.cc:407] Connection header received
0809 02:07:14.847144 (+ 756us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:07:14.847166 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:07:14.847544 (+ 378us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:07:14.847908 (+ 364us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:07:14.848363 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:07:14.849491 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:07:14.850625 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:07:14.851158 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:07:14.851350 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:07:14.852431 (+ 1081us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:07:14.852437 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:07:14.852450 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:07:14.852993 (+ 543us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:07:14.853068 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:07:14.853095 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:07:14.853212 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0809 02:07:14.853403 (+ 191us) server_negotiation.cc:299] Negotiation successful
0809 02:07:14.853676 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":376,"thread_start_us":175,"threads_started":1}
I20260809 02:07:14.856251 25725 raft_consensus.cc:348] T 00000000000000000000000000000000 P 20ab33241766413f961c013481231ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20ab33241766413f961c013481231ec0" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 37037 } }
I20260809 02:07:14.856523 25725 raft_consensus.cc:374] T 00000000000000000000000000000000 P 20ab33241766413f961c013481231ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:07:14.856616 25725 raft_consensus.cc:729] T 00000000000000000000000000000000 P 20ab33241766413f961c013481231ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20ab33241766413f961c013481231ec0, State: Initialized, Role: FOLLOWER
I20260809 02:07:14.857168 25725 consensus_queue.cc:260] T 00000000000000000000000000000000 P 20ab33241766413f961c013481231ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20ab33241766413f961c013481231ec0" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 37037 } }
I20260809 02:07:14.857379 25725 raft_consensus.cc:388] T 00000000000000000000000000000000 P 20ab33241766413f961c013481231ec0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:07:14.857457 25725 raft_consensus.cc:482] T 00000000000000000000000000000000 P 20ab33241766413f961c013481231ec0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:07:14.857601 25725 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 20ab33241766413f961c013481231ec0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:07:14.858336 1795 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 02:07:14.859261 25725 raft_consensus.cc:504] T 00000000000000000000000000000000 P 20ab33241766413f961c013481231ec0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20ab33241766413f961c013481231ec0" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 37037 } }
I20260809 02:07:14.859671 25725 leader_election.cc:302] T 00000000000000000000000000000000 P 20ab33241766413f961c013481231ec0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 20ab33241766413f961c013481231ec0; no voters:
I20260809 02:07:14.860102 25725 leader_election.cc:288] T 00000000000000000000000000000000 P 20ab33241766413f961c013481231ec0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:07:14.860225 25733 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 20ab33241766413f961c013481231ec0 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:07:14.860517 25733 raft_consensus.cc:686] T 00000000000000000000000000000000 P 20ab33241766413f961c013481231ec0 [term 1 LEADER]: Becoming Leader. State: Replica: 20ab33241766413f961c013481231ec0, State: Running, Role: LEADER
I20260809 02:07:14.861521 25733 consensus_queue.cc:237] T 00000000000000000000000000000000 P 20ab33241766413f961c013481231ec0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20ab33241766413f961c013481231ec0" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 37037 } }
I20260809 02:07:14.861651 25725 sys_catalog.cc:564] T 00000000000000000000000000000000 P 20ab33241766413f961c013481231ec0 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:07:14.863759 25735 sys_catalog.cc:455] T 00000000000000000000000000000000 P 20ab33241766413f961c013481231ec0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "20ab33241766413f961c013481231ec0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20ab33241766413f961c013481231ec0" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 37037 } } }
I20260809 02:07:14.863790 25736 sys_catalog.cc:455] T 00000000000000000000000000000000 P 20ab33241766413f961c013481231ec0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 20ab33241766413f961c013481231ec0. Latest consensus state: current_term: 1 leader_uuid: "20ab33241766413f961c013481231ec0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20ab33241766413f961c013481231ec0" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 37037 } } }
I20260809 02:07:14.863929 25736 sys_catalog.cc:458] T 00000000000000000000000000000000 P 20ab33241766413f961c013481231ec0 [sys.catalog]: This master's current role is: LEADER
I20260809 02:07:14.863929 25735 sys_catalog.cc:458] T 00000000000000000000000000000000 P 20ab33241766413f961c013481231ec0 [sys.catalog]: This master's current role is: LEADER
I20260809 02:07:14.867192 25728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:07:14.861000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41761 (local address 127.1.192.254:37037)
0809 02:07:14.861160 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0809 02:07:14.861165 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:07:14.861381 (+ 216us) server_negotiation.cc:407] Connection header received
0809 02:07:14.861540 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:07:14.861543 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:07:14.861601 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:07:14.861703 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:07:14.862032 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:07:14.862880 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:07:14.864996 (+ 2116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:07:14.865364 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:07:14.865524 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:07:14.866636 (+ 1112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:07:14.866640 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:07:14.866643 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:07:14.866660 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:07:14.866692 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:07:14.866706 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:07:14.866790 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 02:07:14.866947 (+ 157us) server_negotiation.cc:299] Negotiation successful
0809 02:07:14.867015 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
W20260809 02:07:14.867301 25751 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 20ab33241766413f961c013481231ec0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 02:07:14.867393 25751 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 02:07:14.868330 25753 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:07:14.869149 25753 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:07:14.875017 25753 catalog_manager.cc:1347] Generated new cluster ID: 264234507d5a4fdf832493a0482079ee
I20260809 02:07:14.875221 25753 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:07:14.899539 25753 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:07:14.900630 25753 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:07:14.905817 25753 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 20ab33241766413f961c013481231ec0: Generated new TSK 0
I20260809 02:07:14.906599 25753 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:07:14.919368 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 25654
2026-08-09T02:07:14Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1571 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786240941968412-1795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:07:14 dist-test-slave-ss4n krb5kdc[25759](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:07:14 dist-test-slave-ss4n krb5kdc[25759](info): set up 2 sockets
Aug 09 02:07:14 dist-test-slave-ss4n krb5kdc[25759](info): commencing operation
krb5kdc: starting...
W20260809 02:07:16.024739 1795 mini_kdc.cc:121] Time spent starting KDC: real 1.081s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786240941968412-1795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786240941968412-1795-0/krb5kdc/test-user.keytab.
Aug 09 02:07:16 dist-test-slave-ss4n krb5kdc[25759](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241236, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:07:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:07:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.192.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.192.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.192.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab.
Entry for principal kudu/127.1.192.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab.
I20260809 02:07:16.174130 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786240941968412-1795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786240941968412-1795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786240941968412-1795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.192.254:36705
--webserver_interface=127.1.192.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab
--principal=kudu/127.1.192.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:40863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.192.254:36705
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:07:16.291339 25772 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:07:16.291699 25772 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:07:16.291782 25772 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:07:16.296154 25772 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:07:16.296264 25772 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:07:16.296305 25772 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:07:16.296366 25772 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:07:16.296409 25772 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:07:16.302205 25772 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:40863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786240941968412-1795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786240941968412-1795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.192.254:36705
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab
--principal=kudu/127.1.192.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.192.254:36705
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.192.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25772
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786240941968412-1795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:07:16.303426 25772 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:07:16.304530 25772 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:07:16.313730 25780 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:07:16.313741 25777 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:07:16.313741 25778 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:07:16.313901 25772 server_base.cc:1034] running on GCE node
I20260809 02:07:16.314738 25772 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:07:16.315981 25772 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:07:16.317165 25772 hybrid_clock.cc:648] HybridClock initialized: now 1786241236317137 us; error 47 us; skew 500 ppm
Aug 09 02:07:16 dist-test-slave-ss4n krb5kdc[25759](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241236, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:07:16.325245 25772 init.cc:377] Logged in from keytab as kudu/127.1.192.254@KRBTEST.COM (short username kudu)
I20260809 02:07:16.326498 25772 webserver.cc:460] Webserver started at http://127.1.192.254:34815/ using document root <none> and password file <none>
I20260809 02:07:16.327170 25772 fs_manager.cc:359] Metadata directory not provided
I20260809 02:07:16.327278 25772 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:07:16.327529 25772 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:07:16.329888 25772 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786240941968412-1795-0/minicluster-data/master-0/data/instance:
uuid: "671fc3c260c64c6eb12b9c49cc270540"
format_stamp: "Formatted at 2026-08-09 02:07:16 on dist-test-slave-ss4n"
server_key: "0da780e5f2040c924d56515e91a9b65a"
server_key_iv: "7c26d50f048d95375fb248ac64a69118"
server_key_version: "encryptionkey@0"
I20260809 02:07:16.330534 25772 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786240941968412-1795-0/minicluster-data/master-0/wal/instance:
uuid: "671fc3c260c64c6eb12b9c49cc270540"
format_stamp: "Formatted at 2026-08-09 02:07:16 on dist-test-slave-ss4n"
server_key: "0da780e5f2040c924d56515e91a9b65a"
server_key_iv: "7c26d50f048d95375fb248ac64a69118"
server_key_version: "encryptionkey@0"
I20260809 02:07:16.334892 25772 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260809 02:07:16.337829 25787 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:07:16.339305 25772 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 02:07:16.339489 25772 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786240941968412-1795-0/minicluster-data/master-0/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786240941968412-1795-0/minicluster-data/master-0/wal
uuid: "671fc3c260c64c6eb12b9c49cc270540"
format_stamp: "Formatted at 2026-08-09 02:07:16 on dist-test-slave-ss4n"
server_key: "0da780e5f2040c924d56515e91a9b65a"
server_key_iv: "7c26d50f048d95375fb248ac64a69118"
server_key_version: "encryptionkey@0"
I20260809 02:07:16.339661 25772 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786240941968412-1795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786240941968412-1795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786240941968412-1795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:07:16.345252 25772 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:07:16.349100 25772 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:07:16.349337 25772 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:07:16.357867 25772 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.254:36705
I20260809 02:07:16.357915 25843 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.254:36705 every 8 connection(s)
I20260809 02:07:16.358960 25772 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
I20260809 02:07:16.362213 25844 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:07:16.362617 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 25772
I20260809 02:07:16.362771 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786240941968412-1795-0/minicluster-data/master-0/wal/instance
I20260809 02:07:16.363124 1795 external_mini_cluster.cc:1428] Setting key 278daacfd82e26b8677c7b74bb839c70
I20260809 02:07:16.369432 25847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:07:16.364829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52121 (local address 127.1.192.254:36705)
0809 02:07:16.365390 (+ 561us) server_negotiation.cc:206] Beginning negotiation
0809 02:07:16.365411 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0809 02:07:16.365456 (+ 45us) server_negotiation.cc:407] Connection header received
0809 02:07:16.366342 (+ 886us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:07:16.366362 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:07:16.366709 (+ 347us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:07:16.367163 (+ 454us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:07:16.368107 (+ 944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:07:16.368112 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:07:16.368126 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:07:16.368138 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:07:16.368200 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:07:16.368222 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:07:16.368392 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0809 02:07:16.368476 (+ 84us) server_negotiation.cc:299] Negotiation successful
0809 02:07:16.368658 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389,"thread_start_us":187,"threads_started":1}
I20260809 02:07:16.370704 25844 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 671fc3c260c64c6eb12b9c49cc270540: Bootstrap starting.
I20260809 02:07:16.373039 1795 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 02:07:16.373533 25844 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 671fc3c260c64c6eb12b9c49cc270540: Neither blocks nor log segments found. Creating new log.
I20260809 02:07:16.374373 25844 log.cc:824] T 00000000000000000000000000000000 P 671fc3c260c64c6eb12b9c49cc270540: Log is configured to *not* fsync() on all Append() calls
I20260809 02:07:16.377012 25844 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 671fc3c260c64c6eb12b9c49cc270540: No bootstrap required, opened a new log
I20260809 02:07:16.377437 25847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:07:16.375524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46793 (local address 127.1.192.254:36705)
0809 02:07:16.375645 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0809 02:07:16.375652 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 02:07:16.375854 (+ 202us) server_negotiation.cc:407] Connection header received
0809 02:07:16.376022 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:07:16.376025 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:07:16.376061 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:07:16.376159 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:07:16.376936 (+ 777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:07:16.376939 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:07:16.376942 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:07:16.376954 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:07:16.376982 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:07:16.377002 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:07:16.377099 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0809 02:07:16.377250 (+ 151us) server_negotiation.cc:299] Negotiation successful
0809 02:07:16.377303 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 02:07:16.381342 25844 raft_consensus.cc:348] T 00000000000000000000000000000000 P 671fc3c260c64c6eb12b9c49cc270540 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "671fc3c260c64c6eb12b9c49cc270540" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 36705 } }
I20260809 02:07:16.381726 25844 raft_consensus.cc:374] T 00000000000000000000000000000000 P 671fc3c260c64c6eb12b9c49cc270540 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:07:16.381775 25844 raft_consensus.cc:729] T 00000000000000000000000000000000 P 671fc3c260c64c6eb12b9c49cc270540 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 671fc3c260c64c6eb12b9c49cc270540, State: Initialized, Role: FOLLOWER
I20260809 02:07:16.382257 25844 consensus_queue.cc:260] T 00000000000000000000000000000000 P 671fc3c260c64c6eb12b9c49cc270540 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "671fc3c260c64c6eb12b9c49cc270540" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 36705 } }
I20260809 02:07:16.382428 25844 raft_consensus.cc:388] T 00000000000000000000000000000000 P 671fc3c260c64c6eb12b9c49cc270540 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:07:16.382508 25844 raft_consensus.cc:482] T 00000000000000000000000000000000 P 671fc3c260c64c6eb12b9c49cc270540 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:07:16.382632 25844 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 671fc3c260c64c6eb12b9c49cc270540 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:07:16.383767 25844 raft_consensus.cc:504] T 00000000000000000000000000000000 P 671fc3c260c64c6eb12b9c49cc270540 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "671fc3c260c64c6eb12b9c49cc270540" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 36705 } }
I20260809 02:07:16.384145 25844 leader_election.cc:302] T 00000000000000000000000000000000 P 671fc3c260c64c6eb12b9c49cc270540 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 671fc3c260c64c6eb12b9c49cc270540; no voters:
I20260809 02:07:16.384505 25844 leader_election.cc:288] T 00000000000000000000000000000000 P 671fc3c260c64c6eb12b9c49cc270540 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:07:16.384759 25855 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 671fc3c260c64c6eb12b9c49cc270540 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:07:16.385025 25855 raft_consensus.cc:686] T 00000000000000000000000000000000 P 671fc3c260c64c6eb12b9c49cc270540 [term 1 LEADER]: Becoming Leader. State: Replica: 671fc3c260c64c6eb12b9c49cc270540, State: Running, Role: LEADER
I20260809 02:07:16.385409 25855 consensus_queue.cc:237] T 00000000000000000000000000000000 P 671fc3c260c64c6eb12b9c49cc270540 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "671fc3c260c64c6eb12b9c49cc270540" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 36705 } }
I20260809 02:07:16.386128 25844 sys_catalog.cc:564] T 00000000000000000000000000000000 P 671fc3c260c64c6eb12b9c49cc270540 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:07:16.387444 25856 sys_catalog.cc:455] T 00000000000000000000000000000000 P 671fc3c260c64c6eb12b9c49cc270540 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "671fc3c260c64c6eb12b9c49cc270540" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "671fc3c260c64c6eb12b9c49cc270540" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 36705 } } }
I20260809 02:07:16.387625 25856 sys_catalog.cc:458] T 00000000000000000000000000000000 P 671fc3c260c64c6eb12b9c49cc270540 [sys.catalog]: This master's current role is: LEADER
I20260809 02:07:16.388047 25857 sys_catalog.cc:455] T 00000000000000000000000000000000 P 671fc3c260c64c6eb12b9c49cc270540 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 671fc3c260c64c6eb12b9c49cc270540. Latest consensus state: current_term: 1 leader_uuid: "671fc3c260c64c6eb12b9c49cc270540" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "671fc3c260c64c6eb12b9c49cc270540" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 36705 } } }
I20260809 02:07:16.388206 25857 sys_catalog.cc:458] T 00000000000000000000000000000000 P 671fc3c260c64c6eb12b9c49cc270540 [sys.catalog]: This master's current role is: LEADER
I20260809 02:07:16.388331 25864 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:07:16.391477 25864 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:07:16.398425 25864 catalog_manager.cc:1347] Generated new cluster ID: 6a4d82e93bd94c0181a15a95438af42a
I20260809 02:07:16.398530 25864 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:07:16.408975 25864 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:07:16.409379 25864 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:07:16.418896 25864 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 671fc3c260c64c6eb12b9c49cc270540: Generated new TSK 0
I20260809 02:07:16.419822 25864 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:07:16.425284 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 25772
2026-08-09T02:07:16Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1504 ms)
[----------] 18 tests from AuthTokenIssuingTest (28202 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786240941968412-1795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:07:16 dist-test-slave-ss4n krb5kdc[25877](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:07:16 dist-test-slave-ss4n krb5kdc[25877](info): set up 2 sockets
Aug 09 02:07:16 dist-test-slave-ss4n krb5kdc[25877](info): commencing operation
krb5kdc: starting...
W20260809 02:07:17.580124 1795 mini_kdc.cc:121] Time spent starting KDC: real 1.132s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786240941968412-1795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786240941968412-1795-0/krb5kdc/test-user.keytab.
Aug 09 02:07:17 dist-test-slave-ss4n krb5kdc[25877](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241237, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:07:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:07:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.192.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.192.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.192.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab.
Entry for principal kudu/127.1.192.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab.
I20260809 02:07:17.721915 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786240941968412-1795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786240941968412-1795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786240941968412-1795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.192.254:44345
--webserver_interface=127.1.192.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab
--principal=kudu/127.1.192.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:38531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.192.254:44345,127.1.192.253:32939,127.1.192.252:37197
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:07:17.839326 25890 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 02:07:17.839737 25890 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:07:17.839809 25890 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:07:17.839897 25890 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:07:17.844079 25890 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:07:17.844206 25890 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:07:17.844242 25890 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:07:17.844282 25890 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:07:17.844316 25890 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:07:17.849880 25890 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:38531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786240941968412-1795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786240941968412-1795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.192.254:44345,127.1.192.253:32939,127.1.192.252:37197
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab
--principal=kudu/127.1.192.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.192.254:44345
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.192.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25890
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786240941968412-1795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:07:17.851100 25890 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:07:17.852185 25890 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:07:17.861320 25896 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:07:17.861330 25900 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:07:17.861321 25897 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:07:17.861711 25890 server_base.cc:1034] running on GCE node
I20260809 02:07:17.862350 25890 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:07:17.863586 25890 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:07:17.864811 25890 hybrid_clock.cc:648] HybridClock initialized: now 1786241237864775 us; error 53 us; skew 500 ppm
Aug 09 02:07:17 dist-test-slave-ss4n krb5kdc[25877](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241237, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:07:17.867930 25890 init.cc:377] Logged in from keytab as kudu/127.1.192.254@KRBTEST.COM (short username kudu)
I20260809 02:07:17.868894 25890 webserver.cc:460] Webserver started at http://127.1.192.254:34511/ using document root <none> and password file <none>
I20260809 02:07:17.869442 25890 fs_manager.cc:359] Metadata directory not provided
I20260809 02:07:17.869589 25890 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:07:17.869818 25890 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:07:17.871984 25890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786240941968412-1795-0/minicluster-data/master-0/data/instance:
uuid: "bfee98e4126949e1b107df21aba96eac"
format_stamp: "Formatted at 2026-08-09 02:07:17 on dist-test-slave-ss4n"
server_key: "91d6ae2a3d6ee06c1fc36c4c53399943"
server_key_iv: "25fbf39e81bec775a9610fe572895f2d"
server_key_version: "encryptionkey@0"
I20260809 02:07:17.872604 25890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786240941968412-1795-0/minicluster-data/master-0/wal/instance:
uuid: "bfee98e4126949e1b107df21aba96eac"
format_stamp: "Formatted at 2026-08-09 02:07:17 on dist-test-slave-ss4n"
server_key: "91d6ae2a3d6ee06c1fc36c4c53399943"
server_key_iv: "25fbf39e81bec775a9610fe572895f2d"
server_key_version: "encryptionkey@0"
I20260809 02:07:17.876569 25890 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.005s
I20260809 02:07:17.879406 25908 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:07:17.880775 25890 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 02:07:17.880923 25890 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786240941968412-1795-0/minicluster-data/master-0/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786240941968412-1795-0/minicluster-data/master-0/wal
uuid: "bfee98e4126949e1b107df21aba96eac"
format_stamp: "Formatted at 2026-08-09 02:07:17 on dist-test-slave-ss4n"
server_key: "91d6ae2a3d6ee06c1fc36c4c53399943"
server_key_iv: "25fbf39e81bec775a9610fe572895f2d"
server_key_version: "encryptionkey@0"
I20260809 02:07:17.881050 25890 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786240941968412-1795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786240941968412-1795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786240941968412-1795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:07:17.895361 25890 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:07:17.899005 25890 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:07:17.899227 25890 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:07:17.907868 25890 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.254:44345
I20260809 02:07:17.908176 25964 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.254:44345 every 8 connection(s)
I20260809 02:07:17.909122 25890 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
I20260809 02:07:17.910542 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 25890
I20260809 02:07:17.910712 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786240941968412-1795-0/minicluster-data/master-0/wal/instance
I20260809 02:07:17.911007 1795 external_mini_cluster.cc:1428] Setting key bbfc84001744ca4635e946667913b369
I20260809 02:07:17.912443 25965 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:07:17.918270 25965 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.192.254" port: 44345 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.1.192.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.192.253@KRBTEST.COM" created.
Aug 09 02:07:17 dist-test-slave-ss4n krb5kdc[25877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241237, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.254@KRBTEST.COM for kudu/127.1.192.254@KRBTEST.COM
I20260809 02:07:17.934993 25968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:07:17.921267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:44345 (local address 127.0.0.1:43898)
0809 02:07:17.921837 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0809 02:07:17.921893 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0809 02:07:17.922891 (+ 998us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:07:17.923640 (+ 749us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:07:17.923645 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:07:17.924093 (+ 448us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:07:17.924408 (+ 315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:07:17.924445 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:07:17.925767 (+ 1322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:07:17.925770 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:07:17.926548 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:07:17.926557 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:07:17.927203 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:07:17.927207 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:07:17.927319 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:07:17.927957 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:07:17.927974 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:07:17.929783 (+ 1809us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:07:17.931882 (+ 2099us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:07:17.931893 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:07:17.931903 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:07:17.932138 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:07:17.932463 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:07:17.932466 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:07:17.932468 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:07:17.932623 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:07:17.933212 (+ 589us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:07:17.933218 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:07:17.933356 (+ 138us) client_negotiation.cc:769] Sending connection context
0809 02:07:17.933606 (+ 250us) client_negotiation.cc:240] Negotiation successful
0809 02:07:17.933979 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":335,"mutex_wait_us":108,"thread_start_us":183,"threads_started":1}
I20260809 02:07:17.934993 25969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:07:17.921803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43898 (local address 127.1.192.254:44345)
0809 02:07:17.922340 (+ 537us) server_negotiation.cc:206] Beginning negotiation
0809 02:07:17.922357 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 02:07:17.922400 (+ 43us) server_negotiation.cc:407] Connection header received
0809 02:07:17.923265 (+ 865us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:07:17.923269 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:07:17.923449 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:07:17.923644 (+ 195us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:07:17.924572 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:07:17.925641 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:07:17.926674 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:07:17.927045 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:07:17.927216 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:07:17.929932 (+ 2716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:07:17.929937 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:07:17.929946 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:07:17.929966 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:07:17.931773 (+ 1807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:07:17.932249 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:07:17.932261 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:07:17.932262 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:07:17.932362 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:07:17.932716 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:07:17.932721 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:07:17.932724 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:07:17.933096 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:07:17.933242 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0809 02:07:17.933674 (+ 432us) server_negotiation.cc:299] Negotiation successful
0809 02:07:17.933923 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":450,"thread_start_us":145,"threads_started":1}
I20260809 02:07:17.937810 25965 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.192.253" port: 32939 } has no permanent_uuid. Determining permanent_uuid...
I20260809 02:07:17.938730 25968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:07:17.938408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.253:32939 (local address 127.0.0.1:48764)
0809 02:07:17.938546 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0809 02:07:17.938628 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.253:32939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260809 02:07:17.938992 25911 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.192.253:32939: connect: Connection refused (error 111)
I20260809 02:07:17.940042 25968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:07:17.939817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.253:32939 (local address 127.0.0.1:48768)
0809 02:07:17.939927 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0809 02:07:17.939981 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.253:32939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260809 02:07:17.940274 25965 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.192.253:32939: Network error: Client connection negotiation failed: client connection to 127.1.192.253:32939: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.192.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.253.keytab.
Entry for principal kudu/127.1.192.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.253.keytab.
I20260809 02:07:17.946281 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786240941968412-1795-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786240941968412-1795-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786240941968412-1795-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786240941968412-1795-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.192.253:32939
--webserver_interface=127.1.192.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.253.keytab
--principal=kudu/127.1.192.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:38531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.192.254:44345,127.1.192.253:32939,127.1.192.252:37197
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 02:07:17.989614 25965 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.192.253" port: 32939 } attempt: 1
I20260809 02:07:17.990739 25968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:07:17.990324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.253:32939 (local address 127.0.0.1:48782)
0809 02:07:17.990572 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0809 02:07:17.990652 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.253:32939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260809 02:07:17.991730 25968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:07:17.991462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.253:32939 (local address 127.0.0.1:48786)
0809 02:07:17.991594 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0809 02:07:17.991663 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.253:32939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260809 02:07:17.991894 25965 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.192.253:32939: Network error: Client connection negotiation failed: client connection to 127.1.192.253:32939: connect: Connection refused (error 111)
I20260809 02:07:18.060086 25965 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.192.253" port: 32939 } attempt: 2
I20260809 02:07:18.060990 25968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:07:18.060668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.253:32939 (local address 127.0.0.1:48792)
0809 02:07:18.060841 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0809 02:07:18.060910 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.253:32939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260809 02:07:18.061901 25968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:07:18.061653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.253:32939 (local address 127.0.0.1:48806)
0809 02:07:18.061774 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0809 02:07:18.061833 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.253:32939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260809 02:07:18.062075 25965 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.192.253:32939: Network error: Client connection negotiation failed: client connection to 127.1.192.253:32939: connect: Connection refused (error 111)
W20260809 02:07:18.064983 25972 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 02:07:18.065325 25972 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:07:18.065389 25972 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:07:18.065485 25972 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:07:18.069685 25972 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:07:18.069767 25972 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:07:18.069815 25972 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:07:18.069872 25972 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:07:18.069909 25972 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:07:18.075567 25972 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:38531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786240941968412-1795-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786240941968412-1795-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.1.192.254:44345,127.1.192.253:32939,127.1.192.252:37197
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.253.keytab
--principal=kudu/127.1.192.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.192.253:32939
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786240941968412-1795-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.1.192.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25972
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786240941968412-1795-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:07:18.076747 25972 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:07:18.077880 25972 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:07:18.087268 25979 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:07:18.087371 25981 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:07:18.087241 25977 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:07:18.087558 25972 server_base.cc:1034] running on GCE node
I20260809 02:07:18.088184 25972 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:07:18.089445 25972 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:07:18.090691 25972 hybrid_clock.cc:648] HybridClock initialized: now 1786241238090665 us; error 39 us; skew 500 ppm
Aug 09 02:07:18 dist-test-slave-ss4n krb5kdc[25877](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241238, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:07:18.093981 25972 init.cc:377] Logged in from keytab as kudu/127.1.192.253@KRBTEST.COM (short username kudu)
I20260809 02:07:18.095043 25972 webserver.cc:460] Webserver started at http://127.1.192.253:35403/ using document root <none> and password file <none>
I20260809 02:07:18.095625 25972 fs_manager.cc:359] Metadata directory not provided
I20260809 02:07:18.095686 25972 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:07:18.095904 25972 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:07:18.098089 25972 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786240941968412-1795-0/minicluster-data/master-1/data/instance:
uuid: "86b298e244bc417f961a6eef09e72f6a"
format_stamp: "Formatted at 2026-08-09 02:07:18 on dist-test-slave-ss4n"
server_key: "71edb7efe08abccda2dcb9a031e3a4fa"
server_key_iv: "de571d68a8136be2b9b193b62254e7f2"
server_key_version: "encryptionkey@0"
I20260809 02:07:18.098670 25972 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786240941968412-1795-0/minicluster-data/master-1/wal/instance:
uuid: "86b298e244bc417f961a6eef09e72f6a"
format_stamp: "Formatted at 2026-08-09 02:07:18 on dist-test-slave-ss4n"
server_key: "71edb7efe08abccda2dcb9a031e3a4fa"
server_key_iv: "de571d68a8136be2b9b193b62254e7f2"
server_key_version: "encryptionkey@0"
I20260809 02:07:18.102659 25972 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 02:07:18.105285 25988 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:07:18.106675 25972 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 02:07:18.106827 25972 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786240941968412-1795-0/minicluster-data/master-1/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786240941968412-1795-0/minicluster-data/master-1/wal
uuid: "86b298e244bc417f961a6eef09e72f6a"
format_stamp: "Formatted at 2026-08-09 02:07:18 on dist-test-slave-ss4n"
server_key: "71edb7efe08abccda2dcb9a031e3a4fa"
server_key_iv: "de571d68a8136be2b9b193b62254e7f2"
server_key_version: "encryptionkey@0"
I20260809 02:07:18.106954 25972 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786240941968412-1795-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786240941968412-1795-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786240941968412-1795-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:07:18.120296 25972 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:07:18.123613 25972 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:07:18.123834 25972 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:07:18.131467 25972 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.253:32939
I20260809 02:07:18.131904 26043 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.253:32939 every 8 connection(s)
I20260809 02:07:18.132490 25972 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786240941968412-1795-0/minicluster-data/master-1/data/info.pb
I20260809 02:07:18.134228 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 25972
I20260809 02:07:18.134370 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786240941968412-1795-0/minicluster-data/master-1/wal/instance
I20260809 02:07:18.134728 1795 external_mini_cluster.cc:1428] Setting key 5bc79dc5caa096e788f6938a1bc98ed0
I20260809 02:07:18.136145 26045 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:07:18.140807 26045 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.192.254" port: 44345 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.1.192.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.192.252@KRBTEST.COM" created.
Aug 09 02:07:18 dist-test-slave-ss4n krb5kdc[25877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241238, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.253@KRBTEST.COM for kudu/127.1.192.254@KRBTEST.COM
I20260809 02:07:18.153245 25965 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.192.253" port: 32939 } attempt: 3
I20260809 02:07:18.158322 25969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:07:18.143055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43906 (local address 127.1.192.254:44345)
0809 02:07:18.143242 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0809 02:07:18.143247 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:07:18.144207 (+ 960us) server_negotiation.cc:407] Connection header received
0809 02:07:18.145258 (+ 1051us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:07:18.145261 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:07:18.145323 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:07:18.145438 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:07:18.146799 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:07:18.147594 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:07:18.149132 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:07:18.149572 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:07:18.149716 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:07:18.152486 (+ 2770us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:07:18.152492 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:07:18.152498 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:07:18.152532 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:07:18.155994 (+ 3462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:07:18.156579 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:07:18.156593 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:07:18.156597 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:07:18.156681 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:07:18.157029 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:07:18.157034 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:07:18.157036 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:07:18.157241 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:07:18.157345 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0809 02:07:18.158036 (+ 691us) server_negotiation.cc:299] Negotiation successful
0809 02:07:18.158161 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 02:07:18.159484 26048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:07:18.143435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:44345 (local address 127.0.0.1:43906)
0809 02:07:18.144028 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0809 02:07:18.144075 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 02:07:18.145024 (+ 949us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:07:18.145652 (+ 628us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:07:18.145658 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:07:18.146076 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:07:18.146594 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:07:18.146633 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:07:18.147752 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:07:18.147758 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:07:18.149001 (+ 1243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:07:18.149011 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:07:18.149744 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:07:18.149747 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:07:18.149897 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:07:18.150546 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:07:18.150567 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:07:18.152290 (+ 1723us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:07:18.156137 (+ 3847us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:07:18.156144 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:07:18.156162 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:07:18.156469 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:07:18.156777 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:07:18.156781 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:07:18.156784 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:07:18.156930 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:07:18.157349 (+ 419us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:07:18.157354 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:07:18.157719 (+ 365us) client_negotiation.cc:769] Sending connection context
0809 02:07:18.157956 (+ 237us) client_negotiation.cc:240] Negotiation successful
0809 02:07:18.158338 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":173,"threads_started":1}
Aug 09 02:07:18 dist-test-slave-ss4n krb5kdc[25877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241237, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.254@KRBTEST.COM for kudu/127.1.192.253@KRBTEST.COM
I20260809 02:07:18.161389 26045 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.192.253" port: 32939 } has no permanent_uuid. Determining permanent_uuid...
I20260809 02:07:18.164613 25968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:07:18.153797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.253:32939 (local address 127.0.0.1:48822)
0809 02:07:18.153957 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0809 02:07:18.153972 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 02:07:18.154073 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:07:18.154761 (+ 688us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:07:18.154764 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:07:18.154957 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:07:18.155130 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:07:18.155142 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:07:18.156241 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:07:18.156248 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:07:18.157163 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:07:18.157174 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:07:18.157955 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:07:18.157959 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:07:18.158079 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:07:18.158752 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:07:18.158801 (+ 49us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:07:18.160360 (+ 1559us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:07:18.162831 (+ 2471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:07:18.162835 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:07:18.162837 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:07:18.163136 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:07:18.163430 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:07:18.163433 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:07:18.163435 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:07:18.163529 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:07:18.164038 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:07:18.164041 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:07:18.164152 (+ 111us) client_negotiation.cc:769] Sending connection context
0809 02:07:18.164281 (+ 129us) client_negotiation.cc:240] Negotiation successful
0809 02:07:18.164432 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260809 02:07:18.164777 26050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:07:18.153912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48822 (local address 127.1.192.253:32939)
0809 02:07:18.154330 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0809 02:07:18.154347 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 02:07:18.154369 (+ 22us) server_negotiation.cc:407] Connection header received
0809 02:07:18.154404 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:07:18.154408 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:07:18.154577 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:07:18.154746 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:07:18.155258 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:07:18.156038 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:07:18.157277 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:07:18.157793 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:07:18.157949 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:07:18.160503 (+ 2554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:07:18.160516 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:07:18.160532 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:07:18.160587 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:07:18.162668 (+ 2081us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:07:18.163254 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:07:18.163258 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:07:18.163259 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:07:18.163330 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:07:18.163625 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:07:18.163630 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:07:18.163634 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:07:18.163921 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:07:18.164076 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0809 02:07:18.164382 (+ 306us) server_negotiation.cc:299] Negotiation successful
0809 02:07:18.164535 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":218,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.192.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.252.keytab.
Entry for principal kudu/127.1.192.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.252.keytab.
I20260809 02:07:18.166811 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786240941968412-1795-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786240941968412-1795-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786240941968412-1795-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786240941968412-1795-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.192.252:37197
--webserver_interface=127.1.192.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.252.keytab
--principal=kudu/127.1.192.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:38531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.192.254:44345,127.1.192.253:32939,127.1.192.252:37197
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 02:07:18 dist-test-slave-ss4n krb5kdc[25877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241238, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.253@KRBTEST.COM for kudu/127.1.192.253@KRBTEST.COM
I20260809 02:07:18.168910 25965 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.192.252" port: 37197 } has no permanent_uuid. Determining permanent_uuid...
I20260809 02:07:18.169884 25968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:07:18.169528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.252:37197 (local address 127.0.0.1:43316)
0809 02:07:18.169732 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0809 02:07:18.169790 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.252:37197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119}
I20260809 02:07:18.170725 25968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:07:18.170394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.252:37197 (local address 127.0.0.1:43330)
0809 02:07:18.170587 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0809 02:07:18.170647 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.252:37197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
W20260809 02:07:18.170912 25965 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.192.252:37197: Network error: Client connection negotiation failed: client connection to 127.1.192.252:37197: connect: Connection refused (error 111)
I20260809 02:07:18.171653 26051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:07:18.162109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48828 (local address 127.1.192.253:32939)
0809 02:07:18.162557 (+ 448us) server_negotiation.cc:206] Beginning negotiation
0809 02:07:18.162567 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 02:07:18.162580 (+ 13us) server_negotiation.cc:407] Connection header received
0809 02:07:18.162621 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:07:18.162622 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:07:18.162656 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:07:18.162753 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:07:18.163132 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:07:18.163729 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:07:18.164537 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:07:18.165426 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:07:18.165675 (+ 249us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:07:18.167866 (+ 2191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:07:18.167873 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:07:18.167879 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:07:18.167910 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:07:18.169761 (+ 1851us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:07:18.170235 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:07:18.170240 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:07:18.170242 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:07:18.170341 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:07:18.170680 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:07:18.170683 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:07:18.170685 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:07:18.170941 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:07:18.171054 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 02:07:18.171383 (+ 329us) server_negotiation.cc:299] Negotiation successful
0809 02:07:18.171513 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":6,"server-negotiator.queue_time_us":288,"thread_start_us":129,"threads_started":1}
I20260809 02:07:18.171883 26048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:07:18.162041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.253:32939 (local address 127.0.0.1:48828)
0809 02:07:18.162168 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0809 02:07:18.162198 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0809 02:07:18.162287 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:07:18.162762 (+ 475us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:07:18.162764 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:07:18.162927 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:07:18.163012 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:07:18.163017 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:07:18.163823 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:07:18.163830 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:07:18.164426 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:07:18.164435 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:07:18.165673 (+ 1238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:07:18.165676 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:07:18.165800 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:07:18.166308 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:07:18.166321 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:07:18.167695 (+ 1374us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:07:18.169862 (+ 2167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:07:18.169868 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:07:18.169870 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:07:18.170138 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:07:18.170459 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:07:18.170463 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:07:18.170466 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:07:18.170562 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:07:18.171051 (+ 489us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:07:18.171056 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:07:18.171185 (+ 129us) client_negotiation.cc:769] Sending connection context
0809 02:07:18.171309 (+ 124us) client_negotiation.cc:240] Negotiation successful
0809 02:07:18.171608 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32,"mutex_wait_us":87}
I20260809 02:07:18.172641 26045 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.192.252" port: 37197 } has no permanent_uuid. Determining permanent_uuid...
I20260809 02:07:18.173432 26048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:07:18.173075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.252:37197 (local address 127.0.0.1:43332)
0809 02:07:18.173244 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0809 02:07:18.173348 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.252:37197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260809 02:07:18.173753 25989 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.192.252:37197: connect: Connection refused (error 111)
I20260809 02:07:18.174685 26048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:07:18.174405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.252:37197 (local address 127.0.0.1:43340)
0809 02:07:18.174542 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0809 02:07:18.174604 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.252:37197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260809 02:07:18.174873 26045 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.192.252:37197: Network error: Client connection negotiation failed: client connection to 127.1.192.252:37197: connect: Connection refused (error 111)
I20260809 02:07:18.202091 25965 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.192.252" port: 37197 } attempt: 1
I20260809 02:07:18.203109 25968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:07:18.202731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.252:37197 (local address 127.0.0.1:43346)
0809 02:07:18.202934 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0809 02:07:18.203008 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.252:37197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260809 02:07:18.203963 25968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:07:18.203719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.252:37197 (local address 127.0.0.1:43350)
0809 02:07:18.203840 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0809 02:07:18.203895 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.252:37197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260809 02:07:18.204104 25965 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.192.252:37197: Network error: Client connection negotiation failed: client connection to 127.1.192.252:37197: connect: Connection refused (error 111)
I20260809 02:07:18.224189 26045 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.192.252" port: 37197 } attempt: 1
I20260809 02:07:18.225234 26048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:07:18.224854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.252:37197 (local address 127.0.0.1:43352)
0809 02:07:18.225043 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0809 02:07:18.225133 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.252:37197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260809 02:07:18.226091 26048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:07:18.225839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.252:37197 (local address 127.0.0.1:43368)
0809 02:07:18.225953 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0809 02:07:18.226007 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.252:37197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260809 02:07:18.226275 26045 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.192.252:37197: Network error: Client connection negotiation failed: client connection to 127.1.192.252:37197: connect: Connection refused (error 111)
I20260809 02:07:18.279335 25965 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.192.252" port: 37197 } attempt: 2
I20260809 02:07:18.280313 25968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:07:18.279943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.252:37197 (local address 127.0.0.1:43384)
0809 02:07:18.280141 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0809 02:07:18.280215 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.252:37197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260809 02:07:18.281222 25968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:07:18.280940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.252:37197 (local address 127.0.0.1:43390)
0809 02:07:18.281090 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0809 02:07:18.281150 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.252:37197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
W20260809 02:07:18.281409 25965 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.192.252:37197: Network error: Client connection negotiation failed: client connection to 127.1.192.252:37197: connect: Connection refused (error 111)
W20260809 02:07:18.288854 26052 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 02:07:18.289186 26052 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:07:18.289243 26052 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:07:18.289321 26052 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:07:18.293670 26052 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:07:18.293741 26052 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:07:18.293761 26052 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:07:18.293778 26052 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:07:18.293831 26052 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:07:18.294468 26045 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.192.252" port: 37197 } attempt: 2
I20260809 02:07:18.295400 26048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:07:18.295035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.252:37197 (local address 127.0.0.1:43402)
0809 02:07:18.295222 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0809 02:07:18.295302 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.252:37197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260809 02:07:18.296255 26048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:07:18.296000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.252:37197 (local address 127.0.0.1:43410)
0809 02:07:18.296129 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0809 02:07:18.296183 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.252:37197: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260809 02:07:18.296451 26045 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.192.252:37197: Network error: Client connection negotiation failed: client connection to 127.1.192.252:37197: connect: Connection refused (error 111)
I20260809 02:07:18.300344 26052 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:38531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786240941968412-1795-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786240941968412-1795-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.1.192.254:44345,127.1.192.253:32939,127.1.192.252:37197
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.252.keytab
--principal=kudu/127.1.192.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.192.252:37197
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786240941968412-1795-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.1.192.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26052
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786240941968412-1795-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:07:18.301612 26052 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:07:18.302703 26052 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:07:18.311926 26058 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:07:18.311930 26061 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:07:18.311995 26059 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:07:18.312096 26052 server_base.cc:1034] running on GCE node
I20260809 02:07:18.312994 26052 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:07:18.314333 26052 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:07:18.315557 26052 hybrid_clock.cc:648] HybridClock initialized: now 1786241238315530 us; error 38 us; skew 500 ppm
Aug 09 02:07:18 dist-test-slave-ss4n krb5kdc[25877](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241238, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:07:18.318885 26052 init.cc:377] Logged in from keytab as kudu/127.1.192.252@KRBTEST.COM (short username kudu)
I20260809 02:07:18.319929 26052 webserver.cc:460] Webserver started at http://127.1.192.252:35393/ using document root <none> and password file <none>
I20260809 02:07:18.320611 26052 fs_manager.cc:359] Metadata directory not provided
I20260809 02:07:18.320722 26052 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:07:18.320972 26052 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:07:18.323424 26052 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786240941968412-1795-0/minicluster-data/master-2/data/instance:
uuid: "4a64508bda0746f1930f1fc924b1484c"
format_stamp: "Formatted at 2026-08-09 02:07:18 on dist-test-slave-ss4n"
server_key: "603ab9c75463f34a3ea1963b06295310"
server_key_iv: "8eb5d4238968e2c422bc9cca19b9b67f"
server_key_version: "encryptionkey@0"
I20260809 02:07:18.324077 26052 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786240941968412-1795-0/minicluster-data/master-2/wal/instance:
uuid: "4a64508bda0746f1930f1fc924b1484c"
format_stamp: "Formatted at 2026-08-09 02:07:18 on dist-test-slave-ss4n"
server_key: "603ab9c75463f34a3ea1963b06295310"
server_key_iv: "8eb5d4238968e2c422bc9cca19b9b67f"
server_key_version: "encryptionkey@0"
I20260809 02:07:18.328450 26052 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260809 02:07:18.331383 26068 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:07:18.332883 26052 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 02:07:18.333076 26052 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786240941968412-1795-0/minicluster-data/master-2/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786240941968412-1795-0/minicluster-data/master-2/wal
uuid: "4a64508bda0746f1930f1fc924b1484c"
format_stamp: "Formatted at 2026-08-09 02:07:18 on dist-test-slave-ss4n"
server_key: "603ab9c75463f34a3ea1963b06295310"
server_key_iv: "8eb5d4238968e2c422bc9cca19b9b67f"
server_key_version: "encryptionkey@0"
I20260809 02:07:18.333222 26052 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786240941968412-1795-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786240941968412-1795-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786240941968412-1795-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:07:18.349442 26052 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:07:18.352903 26052 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:07:18.353148 26052 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:07:18.360948 26052 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.252:37197
I20260809 02:07:18.361407 26123 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.252:37197 every 8 connection(s)
I20260809 02:07:18.362088 26052 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786240941968412-1795-0/minicluster-data/master-2/data/info.pb
I20260809 02:07:18.365087 26124 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:07:18.366452 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 26052
I20260809 02:07:18.366668 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786240941968412-1795-0/minicluster-data/master-2/wal/instance
I20260809 02:07:18.366977 1795 external_mini_cluster.cc:1428] Setting key 4a1093ed7e49d960148bbc112c03793a
I20260809 02:07:18.369987 26124 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.192.254" port: 44345 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 02:07:18 dist-test-slave-ss4n krb5kdc[25877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241237, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.192.252@KRBTEST.COM
Aug 09 02:07:18 dist-test-slave-ss4n krb5kdc[25877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241238, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.252@KRBTEST.COM for kudu/127.1.192.254@KRBTEST.COM
I20260809 02:07:18.380759 25965 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.192.252" port: 37197 } attempt: 3
I20260809 02:07:18.384142 25969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:07:18.372130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43918 (local address 127.1.192.254:44345)
0809 02:07:18.372332 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0809 02:07:18.372338 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:07:18.372713 (+ 375us) server_negotiation.cc:407] Connection header received
0809 02:07:18.372840 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:07:18.372843 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:07:18.372911 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:07:18.372983 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:07:18.373877 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:07:18.374634 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:07:18.375904 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:07:18.376344 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:07:18.376502 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:07:18.379507 (+ 3005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:07:18.379514 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:07:18.379521 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:07:18.379551 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:07:18.381415 (+ 1864us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:07:18.382239 (+ 824us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:07:18.382243 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:07:18.382244 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:07:18.382339 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:07:18.382829 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:07:18.382832 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:07:18.382834 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:07:18.383076 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:07:18.383148 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 02:07:18.383830 (+ 682us) server_negotiation.cc:299] Negotiation successful
0809 02:07:18.383969 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 02:07:18.385803 26129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:07:18.372068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:44345 (local address 127.0.0.1:43918)
0809 02:07:18.372564 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0809 02:07:18.372616 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 02:07:18.372744 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:07:18.373057 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:07:18.373062 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:07:18.373609 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:07:18.373738 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:07:18.373745 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:07:18.374753 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:07:18.374756 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:07:18.375761 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:07:18.375771 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:07:18.376532 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:07:18.376540 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:07:18.376632 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:07:18.377194 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:07:18.377211 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:07:18.379330 (+ 2119us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:07:18.381675 (+ 2345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:07:18.381687 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:07:18.381698 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:07:18.382084 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:07:18.382514 (+ 430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:07:18.382516 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:07:18.382518 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:07:18.382722 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:07:18.383232 (+ 510us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:07:18.383240 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:07:18.383570 (+ 330us) client_negotiation.cc:769] Sending connection context
0809 02:07:18.384294 (+ 724us) client_negotiation.cc:240] Negotiation successful
0809 02:07:18.384606 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":313,"mutex_wait_us":41,"thread_start_us":201,"threads_started":1}
I20260809 02:07:18.386101 26128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:07:18.368637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43416 (local address 127.1.192.252:37197)
0809 02:07:18.369105 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0809 02:07:18.369126 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0809 02:07:18.369171 (+ 45us) server_negotiation.cc:407] Connection header received
0809 02:07:18.370082 (+ 911us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:07:18.370108 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:07:18.370558 (+ 450us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:07:18.370959 (+ 401us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:07:18.371825 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:07:18.373336 (+ 1511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:07:18.374746 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:07:18.375486 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:07:18.375757 (+ 271us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:07:18.379093 (+ 3336us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:07:18.379104 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:07:18.379127 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:07:18.379154 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:07:18.382104 (+ 2950us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:07:18.382665 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:07:18.382672 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:07:18.382674 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:07:18.382769 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:07:18.383132 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:07:18.383137 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:07:18.383139 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:07:18.383573 (+ 434us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:07:18.383861 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0809 02:07:18.384053 (+ 192us) server_negotiation.cc:299] Negotiation successful
0809 02:07:18.384724 (+ 671us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":70,"server-negotiator.queue_time_us":312,"thread_start_us":126,"threads_started":1}
Aug 09 02:07:18 dist-test-slave-ss4n krb5kdc[25877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241237, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.254@KRBTEST.COM for kudu/127.1.192.252@KRBTEST.COM
I20260809 02:07:18.387432 26124 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.192.253" port: 32939 } has no permanent_uuid. Determining permanent_uuid...
I20260809 02:07:18.387633 26045 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.192.252" port: 37197 } attempt: 3
I20260809 02:07:18.393642 26130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:07:18.381529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43432 (local address 127.1.192.252:37197)
0809 02:07:18.381901 (+ 372us) server_negotiation.cc:206] Beginning negotiation
0809 02:07:18.381915 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 02:07:18.381933 (+ 18us) server_negotiation.cc:407] Connection header received
0809 02:07:18.381980 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:07:18.381985 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:07:18.382056 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:07:18.382164 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:07:18.382647 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:07:18.383431 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:07:18.384587 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:07:18.384956 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:07:18.385072 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:07:18.387702 (+ 2630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:07:18.387706 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:07:18.387709 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:07:18.387736 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:07:18.391184 (+ 3448us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:07:18.391755 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:07:18.391759 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:07:18.391761 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:07:18.391877 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:07:18.392494 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:07:18.392498 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:07:18.392500 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:07:18.392731 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:07:18.392848 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0809 02:07:18.393228 (+ 380us) server_negotiation.cc:299] Negotiation successful
0809 02:07:18.393365 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":158,"threads_started":1}
Aug 09 02:07:18 dist-test-slave-ss4n krb5kdc[25877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241238, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.253@KRBTEST.COM for kudu/127.1.192.252@KRBTEST.COM
I20260809 02:07:18.393834 25968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:07:18.381423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.252:37197 (local address 127.0.0.1:43432)
0809 02:07:18.381620 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0809 02:07:18.381637 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 02:07:18.381714 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:07:18.382218 (+ 504us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:07:18.382224 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:07:18.382407 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:07:18.382525 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:07:18.382532 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:07:18.383552 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:07:18.383556 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:07:18.384463 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:07:18.384473 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:07:18.385087 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:07:18.385091 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:07:18.385182 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:07:18.385741 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:07:18.385757 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:07:18.387534 (+ 1777us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:07:18.391312 (+ 3778us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:07:18.391315 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:07:18.391318 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:07:18.391619 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:07:18.391970 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:07:18.391973 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:07:18.391975 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:07:18.392086 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:07:18.392858 (+ 772us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:07:18.392863 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:07:18.393000 (+ 137us) client_negotiation.cc:769] Sending connection context
0809 02:07:18.393135 (+ 135us) client_negotiation.cc:240] Negotiation successful
0809 02:07:18.393633 (+ 498us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":83}
Aug 09 02:07:18 dist-test-slave-ss4n krb5kdc[25877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241238, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.252@KRBTEST.COM for kudu/127.1.192.253@KRBTEST.COM
Aug 09 02:07:18 dist-test-slave-ss4n krb5kdc[25877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241237, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.192.254@KRBTEST.COM
I20260809 02:07:18.397948 25965 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bfee98e4126949e1b107df21aba96eac: Bootstrap starting.
I20260809 02:07:18.398836 26048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:07:18.388370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.252:37197 (local address 127.0.0.1:43446)
0809 02:07:18.388518 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0809 02:07:18.388531 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:07:18.388595 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:07:18.389139 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:07:18.389144 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:07:18.389348 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:07:18.389511 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:07:18.389520 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:07:18.390371 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:07:18.390374 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:07:18.391193 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:07:18.391203 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:07:18.391799 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:07:18.391802 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:07:18.391979 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:07:18.392472 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:07:18.392490 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:07:18.394128 (+ 1638us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:07:18.396669 (+ 2541us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:07:18.396674 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:07:18.396677 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:07:18.396985 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:07:18.397332 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:07:18.397336 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:07:18.397340 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:07:18.397453 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:07:18.398296 (+ 843us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:07:18.398299 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:07:18.398407 (+ 108us) client_negotiation.cc:769] Sending connection context
0809 02:07:18.398515 (+ 108us) client_negotiation.cc:240] Negotiation successful
0809 02:07:18.398667 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96,"mutex_wait_us":79}
I20260809 02:07:18.398964 26128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:07:18.388725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43446 (local address 127.1.192.252:37197)
0809 02:07:18.388873 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0809 02:07:18.388877 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:07:18.388890 (+ 13us) server_negotiation.cc:407] Connection header received
0809 02:07:18.388929 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:07:18.388932 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:07:18.388978 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:07:18.389637 (+ 659us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:07:18.389711 (+ 74us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:07:18.390260 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:07:18.391330 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:07:18.391671 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:07:18.391845 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:07:18.394283 (+ 2438us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:07:18.394287 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:07:18.394289 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:07:18.394318 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:07:18.396530 (+ 2212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:07:18.397110 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:07:18.397113 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:07:18.397115 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:07:18.397206 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:07:18.397638 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:07:18.397642 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:07:18.397644 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:07:18.397949 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:07:18.398073 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0809 02:07:18.398637 (+ 564us) server_negotiation.cc:299] Negotiation successful
0809 02:07:18.398777 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 02:07:18.400862 25969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:07:18.389067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43930 (local address 127.1.192.254:44345)
0809 02:07:18.389191 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0809 02:07:18.389198 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 02:07:18.389213 (+ 15us) server_negotiation.cc:407] Connection header received
0809 02:07:18.389314 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:07:18.389319 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:07:18.389366 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:07:18.389454 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:07:18.390258 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:07:18.391134 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:07:18.392362 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:07:18.392732 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:07:18.392846 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:07:18.396989 (+ 4143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:07:18.397004 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:07:18.397011 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:07:18.397049 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:07:18.398903 (+ 1854us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:07:18.399456 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:07:18.399461 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:07:18.399463 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:07:18.399568 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:07:18.399916 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:07:18.399919 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:07:18.399921 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:07:18.400147 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:07:18.400324 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0809 02:07:18.400592 (+ 268us) server_negotiation.cc:299] Negotiation successful
0809 02:07:18.400716 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 02:07:18.401094 25965 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bfee98e4126949e1b107df21aba96eac: Neither blocks nor log segments found. Creating new log.
I20260809 02:07:18.402369 26045 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86b298e244bc417f961a6eef09e72f6a: Bootstrap starting.
I20260809 02:07:18.402372 25965 log.cc:824] T 00000000000000000000000000000000 P bfee98e4126949e1b107df21aba96eac: Log is configured to *not* fsync() on all Append() calls
I20260809 02:07:18.404727 25965 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bfee98e4126949e1b107df21aba96eac: No bootstrap required, opened a new log
I20260809 02:07:18.405254 26045 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 86b298e244bc417f961a6eef09e72f6a: Neither blocks nor log segments found. Creating new log.
I20260809 02:07:18.406226 26045 log.cc:824] T 00000000000000000000000000000000 P 86b298e244bc417f961a6eef09e72f6a: Log is configured to *not* fsync() on all Append() calls
I20260809 02:07:18.407358 26051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:07:18.388303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48832 (local address 127.1.192.253:32939)
0809 02:07:18.388427 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0809 02:07:18.388432 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:07:18.388457 (+ 25us) server_negotiation.cc:407] Connection header received
0809 02:07:18.388497 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:07:18.388500 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:07:18.388553 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:07:18.388630 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:07:18.389272 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:07:18.390182 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:07:18.391330 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:07:18.391688 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:07:18.391773 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:07:18.403257 (+ 11484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:07:18.403264 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:07:18.403270 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:07:18.403299 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:07:18.405193 (+ 1894us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:07:18.405767 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:07:18.405770 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:07:18.405772 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:07:18.405863 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:07:18.406227 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:07:18.406230 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:07:18.406232 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:07:18.406508 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:07:18.406583 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 02:07:18.407018 (+ 435us) server_negotiation.cc:299] Negotiation successful
0809 02:07:18.407149 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":52}
I20260809 02:07:18.407861 26129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:07:18.388038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.253:32939 (local address 127.0.0.1:48832)
0809 02:07:18.388191 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 02:07:18.388209 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 02:07:18.388291 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:07:18.388821 (+ 530us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:07:18.388825 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:07:18.389008 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:07:18.389120 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:07:18.389127 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:07:18.390345 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:07:18.390348 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:07:18.391193 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:07:18.391203 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:07:18.392236 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:07:18.392240 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:07:18.392997 (+ 757us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:07:18.393546 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:07:18.393561 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:07:18.403074 (+ 9513us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:07:18.405319 (+ 2245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:07:18.405323 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:07:18.405325 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:07:18.405641 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:07:18.406005 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:07:18.406011 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:07:18.406016 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:07:18.406113 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:07:18.406661 (+ 548us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:07:18.406667 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:07:18.406776 (+ 109us) client_negotiation.cc:769] Sending connection context
0809 02:07:18.407501 (+ 725us) client_negotiation.cc:240] Negotiation successful
0809 02:07:18.407686 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260809 02:07:18.408812 26045 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86b298e244bc417f961a6eef09e72f6a: No bootstrap required, opened a new log
I20260809 02:07:18.409184 26124 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.192.252" port: 37197 } has no permanent_uuid. Determining permanent_uuid...
I20260809 02:07:18.409924 25965 raft_consensus.cc:348] T 00000000000000000000000000000000 P bfee98e4126949e1b107df21aba96eac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfee98e4126949e1b107df21aba96eac" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 44345 } } peers { permanent_uuid: "86b298e244bc417f961a6eef09e72f6a" member_type: VOTER last_known_addr { host: "127.1.192.253" port: 32939 } } peers { permanent_uuid: "4a64508bda0746f1930f1fc924b1484c" member_type: VOTER last_known_addr { host: "127.1.192.252" port: 37197 } }
I20260809 02:07:18.410311 25965 raft_consensus.cc:729] T 00000000000000000000000000000000 P bfee98e4126949e1b107df21aba96eac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfee98e4126949e1b107df21aba96eac, State: Initialized, Role: FOLLOWER
I20260809 02:07:18.410732 25965 consensus_queue.cc:260] T 00000000000000000000000000000000 P bfee98e4126949e1b107df21aba96eac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfee98e4126949e1b107df21aba96eac" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 44345 } } peers { permanent_uuid: "86b298e244bc417f961a6eef09e72f6a" member_type: VOTER last_known_addr { host: "127.1.192.253" port: 32939 } } peers { permanent_uuid: "4a64508bda0746f1930f1fc924b1484c" member_type: VOTER last_known_addr { host: "127.1.192.252" port: 37197 } }
Aug 09 02:07:18 dist-test-slave-ss4n krb5kdc[25877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241237, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.192.253@KRBTEST.COM
I20260809 02:07:18.412532 25965 sys_catalog.cc:564] T 00000000000000000000000000000000 P bfee98e4126949e1b107df21aba96eac [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:07:18.413609 26045 raft_consensus.cc:348] T 00000000000000000000000000000000 P 86b298e244bc417f961a6eef09e72f6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfee98e4126949e1b107df21aba96eac" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 44345 } } peers { permanent_uuid: "86b298e244bc417f961a6eef09e72f6a" member_type: VOTER last_known_addr { host: "127.1.192.253" port: 32939 } } peers { permanent_uuid: "4a64508bda0746f1930f1fc924b1484c" member_type: VOTER last_known_addr { host: "127.1.192.252" port: 37197 } }
I20260809 02:07:18.413919 26045 raft_consensus.cc:729] T 00000000000000000000000000000000 P 86b298e244bc417f961a6eef09e72f6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86b298e244bc417f961a6eef09e72f6a, State: Initialized, Role: FOLLOWER
I20260809 02:07:18.414335 26045 consensus_queue.cc:260] T 00000000000000000000000000000000 P 86b298e244bc417f961a6eef09e72f6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfee98e4126949e1b107df21aba96eac" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 44345 } } peers { permanent_uuid: "86b298e244bc417f961a6eef09e72f6a" member_type: VOTER last_known_addr { host: "127.1.192.253" port: 32939 } } peers { permanent_uuid: "4a64508bda0746f1930f1fc924b1484c" member_type: VOTER last_known_addr { host: "127.1.192.252" port: 37197 } }
I20260809 02:07:18.416082 26045 sys_catalog.cc:564] T 00000000000000000000000000000000 P 86b298e244bc417f961a6eef09e72f6a [sys.catalog]: configured and running, proceeding with master startup.
Aug 09 02:07:18 dist-test-slave-ss4n krb5kdc[25877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241238, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.252@KRBTEST.COM for kudu/127.1.192.252@KRBTEST.COM
I20260809 02:07:18.420192 26129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:07:18.409899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.252:37197 (local address 127.0.0.1:43456)
0809 02:07:18.410678 (+ 779us) negotiation.cc:107] Waiting for socket to connect
0809 02:07:18.410696 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 02:07:18.410810 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:07:18.411238 (+ 428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:07:18.411243 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:07:18.411507 (+ 264us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:07:18.411630 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:07:18.411637 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:07:18.412714 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:07:18.412717 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:07:18.413571 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:07:18.413585 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:07:18.414303 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:07:18.414308 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:07:18.414422 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:07:18.414855 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:07:18.414875 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:07:18.416551 (+ 1676us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:07:18.418581 (+ 2030us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:07:18.418584 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:07:18.418586 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:07:18.418841 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:07:18.419122 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:07:18.419125 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:07:18.419127 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:07:18.419205 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:07:18.419663 (+ 458us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:07:18.419667 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:07:18.419760 (+ 93us) client_negotiation.cc:769] Sending connection context
0809 02:07:18.419855 (+ 95us) client_negotiation.cc:240] Negotiation successful
0809 02:07:18.420008 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":688}
I20260809 02:07:18.420763 26128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:07:18.410168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43456 (local address 127.1.192.252:37197)
0809 02:07:18.410280 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0809 02:07:18.410285 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:07:18.410746 (+ 461us) server_negotiation.cc:407] Connection header received
0809 02:07:18.410910 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:07:18.410913 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:07:18.410957 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:07:18.411008 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:07:18.411744 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:07:18.412554 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:07:18.413691 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:07:18.414053 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:07:18.414136 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:07:18.416699 (+ 2563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:07:18.416704 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:07:18.416710 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:07:18.416741 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:07:18.418471 (+ 1730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:07:18.418948 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:07:18.418951 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:07:18.418953 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:07:18.419031 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:07:18.419292 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:07:18.419294 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:07:18.419296 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:07:18.419530 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:07:18.419626 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0809 02:07:18.420469 (+ 843us) server_negotiation.cc:299] Negotiation successful
0809 02:07:18.420585 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 02:07:18.421311 26139 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86b298e244bc417f961a6eef09e72f6a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfee98e4126949e1b107df21aba96eac" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 44345 } } peers { permanent_uuid: "86b298e244bc417f961a6eef09e72f6a" member_type: VOTER last_known_addr { host: "127.1.192.253" port: 32939 } } peers { permanent_uuid: "4a64508bda0746f1930f1fc924b1484c" member_type: VOTER last_known_addr { host: "127.1.192.252" port: 37197 } } }
I20260809 02:07:18.421468 26139 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86b298e244bc417f961a6eef09e72f6a [sys.catalog]: This master's current role is: FOLLOWER
I20260809 02:07:18.420744 26133 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfee98e4126949e1b107df21aba96eac [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfee98e4126949e1b107df21aba96eac" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 44345 } } peers { permanent_uuid: "86b298e244bc417f961a6eef09e72f6a" member_type: VOTER last_known_addr { host: "127.1.192.253" port: 32939 } } peers { permanent_uuid: "4a64508bda0746f1930f1fc924b1484c" member_type: VOTER last_known_addr { host: "127.1.192.252" port: 37197 } } }
I20260809 02:07:18.421865 26133 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfee98e4126949e1b107df21aba96eac [sys.catalog]: This master's current role is: FOLLOWER
W20260809 02:07:18.422259 26145 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bfee98e4126949e1b107df21aba96eac: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 02:07:18.422384 26145 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 02:07:18.423523 26124 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a64508bda0746f1930f1fc924b1484c: Bootstrap starting.
I20260809 02:07:18.427062 26050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:07:18.406411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48848 (local address 127.1.192.253:32939)
0809 02:07:18.406535 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0809 02:07:18.406540 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:07:18.406559 (+ 19us) server_negotiation.cc:407] Connection header received
0809 02:07:18.406722 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:07:18.406725 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:07:18.406782 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:07:18.406853 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:07:18.407811 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:07:18.408396 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:07:18.409463 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:07:18.409858 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:07:18.409965 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:07:18.413234 (+ 3269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:07:18.413238 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:07:18.413241 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:07:18.413272 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:07:18.415751 (+ 2479us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:07:18.416803 (+ 1052us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:07:18.416807 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:07:18.416808 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:07:18.416902 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:07:18.417905 (+ 1003us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:07:18.417911 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:07:18.417914 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:07:18.425903 (+ 7989us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:07:18.426154 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0809 02:07:18.426701 (+ 547us) server_negotiation.cc:299] Negotiation successful
0809 02:07:18.426898 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
W20260809 02:07:18.428026 26156 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 86b298e244bc417f961a6eef09e72f6a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 02:07:18.428188 26156 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 02:07:18.429505 26124 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4a64508bda0746f1930f1fc924b1484c: Neither blocks nor log segments found. Creating new log.
I20260809 02:07:18.430529 26124 log.cc:824] T 00000000000000000000000000000000 P 4a64508bda0746f1930f1fc924b1484c: Log is configured to *not* fsync() on all Append() calls
I20260809 02:07:18.432469 1795 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 02:07:18.433992 25936 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "bfee98e4126949e1b107df21aba96eac"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43930
I20260809 02:07:18.434353 25936 raft_consensus.cc:482] T 00000000000000000000000000000000 P bfee98e4126949e1b107df21aba96eac [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 02:07:18.434501 25936 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bfee98e4126949e1b107df21aba96eac [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:07:18.431917 25890 master_runner.cc:186] Error getting master registration for 127.1.192.252:37197: OK, instance_id { permanent_uuid: "4a64508bda0746f1930f1fc924b1484c" instance_seqno: 1786241238358688 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 02:07:18.436057 25936 raft_consensus.cc:504] T 00000000000000000000000000000000 P bfee98e4126949e1b107df21aba96eac [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfee98e4126949e1b107df21aba96eac" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 44345 } } peers { permanent_uuid: "86b298e244bc417f961a6eef09e72f6a" member_type: VOTER last_known_addr { host: "127.1.192.253" port: 32939 } } peers { permanent_uuid: "4a64508bda0746f1930f1fc924b1484c" member_type: VOTER last_known_addr { host: "127.1.192.252" port: 37197 } }
I20260809 02:07:18.436213 25890 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 02:07:18.437832 26016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bfee98e4126949e1b107df21aba96eac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "86b298e244bc417f961a6eef09e72f6a"
I20260809 02:07:18.438201 26016 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 86b298e244bc417f961a6eef09e72f6a [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:07:18.439056 26124 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a64508bda0746f1930f1fc924b1484c: No bootstrap required, opened a new log
I20260809 02:07:18.439670 26016 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 86b298e244bc417f961a6eef09e72f6a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bfee98e4126949e1b107df21aba96eac in term 1.
I20260809 02:07:18.440223 25911 leader_election.cc:302] T 00000000000000000000000000000000 P bfee98e4126949e1b107df21aba96eac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 86b298e244bc417f961a6eef09e72f6a, bfee98e4126949e1b107df21aba96eac; no voters:
I20260809 02:07:18.440625 26133 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bfee98e4126949e1b107df21aba96eac [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:07:18.440936 26133 raft_consensus.cc:686] T 00000000000000000000000000000000 P bfee98e4126949e1b107df21aba96eac [term 1 LEADER]: Becoming Leader. State: Replica: bfee98e4126949e1b107df21aba96eac, State: Running, Role: LEADER
I20260809 02:07:18.441322 26133 consensus_queue.cc:237] T 00000000000000000000000000000000 P bfee98e4126949e1b107df21aba96eac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfee98e4126949e1b107df21aba96eac" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 44345 } } peers { permanent_uuid: "86b298e244bc417f961a6eef09e72f6a" member_type: VOTER last_known_addr { host: "127.1.192.253" port: 32939 } } peers { permanent_uuid: "4a64508bda0746f1930f1fc924b1484c" member_type: VOTER last_known_addr { host: "127.1.192.252" port: 37197 } }
I20260809 02:07:18.443548 26133 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfee98e4126949e1b107df21aba96eac [sys.catalog]: SysCatalogTable state changed. Reason: New leader bfee98e4126949e1b107df21aba96eac. Latest consensus state: current_term: 1 leader_uuid: "bfee98e4126949e1b107df21aba96eac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfee98e4126949e1b107df21aba96eac" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 44345 } } peers { permanent_uuid: "86b298e244bc417f961a6eef09e72f6a" member_type: VOTER last_known_addr { host: "127.1.192.253" port: 32939 } } peers { permanent_uuid: "4a64508bda0746f1930f1fc924b1484c" member_type: VOTER last_known_addr { host: "127.1.192.252" port: 37197 } } }
I20260809 02:07:18.443722 26133 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfee98e4126949e1b107df21aba96eac [sys.catalog]: This master's current role is: LEADER
I20260809 02:07:18.444221 26124 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4a64508bda0746f1930f1fc924b1484c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfee98e4126949e1b107df21aba96eac" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 44345 } } peers { permanent_uuid: "86b298e244bc417f961a6eef09e72f6a" member_type: VOTER last_known_addr { host: "127.1.192.253" port: 32939 } } peers { permanent_uuid: "4a64508bda0746f1930f1fc924b1484c" member_type: VOTER last_known_addr { host: "127.1.192.252" port: 37197 } }
I20260809 02:07:18.444571 26124 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4a64508bda0746f1930f1fc924b1484c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a64508bda0746f1930f1fc924b1484c, State: Initialized, Role: FOLLOWER
I20260809 02:07:18.444774 25936 leader_election.cc:288] T 00000000000000000000000000000000 P bfee98e4126949e1b107df21aba96eac [CANDIDATE]: Term 1 election: Requested vote from peers 86b298e244bc417f961a6eef09e72f6a (127.1.192.253:32939), 4a64508bda0746f1930f1fc924b1484c (127.1.192.252:37197)
I20260809 02:07:18.445029 26124 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4a64508bda0746f1930f1fc924b1484c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfee98e4126949e1b107df21aba96eac" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 44345 } } peers { permanent_uuid: "86b298e244bc417f961a6eef09e72f6a" member_type: VOTER last_known_addr { host: "127.1.192.253" port: 32939 } } peers { permanent_uuid: "4a64508bda0746f1930f1fc924b1484c" member_type: VOTER last_known_addr { host: "127.1.192.252" port: 37197 } }
I20260809 02:07:18.446760 26124 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4a64508bda0746f1930f1fc924b1484c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:07:18.447146 26097 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bfee98e4126949e1b107df21aba96eac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "4a64508bda0746f1930f1fc924b1484c"
I20260809 02:07:18.447454 26097 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4a64508bda0746f1930f1fc924b1484c [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:07:18.449007 26097 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4a64508bda0746f1930f1fc924b1484c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bfee98e4126949e1b107df21aba96eac in term 1.
I20260809 02:07:18.449832 26163 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:07:18.451148 26163 catalog_manager.cc:1484] Initializing Kudu cluster ID...
W20260809 02:07:18.456197 26180 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4a64508bda0746f1930f1fc924b1484c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260809 02:07:18.430393 25972 master_runner.cc:186] Error getting master registration for 127.1.192.252:37197: OK, instance_id { permanent_uuid: "4a64508bda0746f1930f1fc924b1484c" instance_seqno: 1786241238358688 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
W20260809 02:07:18.456324 26180 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 02:07:18.456398 25972 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 02:07:18.456736 26164 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a64508bda0746f1930f1fc924b1484c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfee98e4126949e1b107df21aba96eac" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 44345 } } peers { permanent_uuid: "86b298e244bc417f961a6eef09e72f6a" member_type: VOTER last_known_addr { host: "127.1.192.253" port: 32939 } } peers { permanent_uuid: "4a64508bda0746f1930f1fc924b1484c" member_type: VOTER last_known_addr { host: "127.1.192.252" port: 37197 } } }
I20260809 02:07:18.456897 26164 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a64508bda0746f1930f1fc924b1484c [sys.catalog]: This master's current role is: FOLLOWER
I20260809 02:07:18.468691 25969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:07:18.450137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43934 (local address 127.1.192.254:44345)
0809 02:07:18.455842 (+ 5705us) server_negotiation.cc:206] Beginning negotiation
0809 02:07:18.455847 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:07:18.455862 (+ 15us) server_negotiation.cc:407] Connection header received
0809 02:07:18.455904 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:07:18.455907 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:07:18.455965 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:07:18.456043 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:07:18.457437 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:07:18.458521 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:07:18.460000 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:07:18.460441 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:07:18.460530 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:07:18.462996 (+ 2466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:07:18.463001 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:07:18.463004 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:07:18.463046 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:07:18.465331 (+ 2285us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:07:18.467060 (+ 1729us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:07:18.467065 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:07:18.467067 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:07:18.467189 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:07:18.467529 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:07:18.467533 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:07:18.467535 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:07:18.467792 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:07:18.467933 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0809 02:07:18.468343 (+ 410us) server_negotiation.cc:299] Negotiation successful
0809 02:07:18.468495 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5613}
I20260809 02:07:18.470624 26050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:07:18.451870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48864 (local address 127.1.192.253:32939)
0809 02:07:18.452002 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0809 02:07:18.452009 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 02:07:18.456055 (+ 4046us) server_negotiation.cc:407] Connection header received
0809 02:07:18.456368 (+ 313us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:07:18.456373 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:07:18.456434 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:07:18.456498 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:07:18.460910 (+ 4412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:07:18.461813 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:07:18.464402 (+ 2589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:07:18.464761 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:07:18.464840 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:07:18.466734 (+ 1894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:07:18.466739 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:07:18.466742 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:07:18.466774 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:07:18.468508 (+ 1734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:07:18.469030 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:07:18.469034 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:07:18.469036 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:07:18.469150 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:07:18.469511 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:07:18.469515 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:07:18.469517 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:07:18.469782 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:07:18.469914 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0809 02:07:18.470289 (+ 375us) server_negotiation.cc:299] Negotiation successful
0809 02:07:18.470421 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 02:07:18.474725 26128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:07:18.450864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43458 (local address 127.1.192.252:37197)
0809 02:07:18.450991 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0809 02:07:18.450996 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:07:18.455823 (+ 4827us) server_negotiation.cc:407] Connection header received
0809 02:07:18.456675 (+ 852us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:07:18.456679 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:07:18.456732 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:07:18.456790 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:07:18.458719 (+ 1929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:07:18.459570 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:07:18.463346 (+ 3776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:07:18.463743 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:07:18.463840 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:07:18.470607 (+ 6767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:07:18.470612 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:07:18.470626 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:07:18.470663 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:07:18.472626 (+ 1963us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:07:18.473174 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:07:18.473178 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:07:18.473180 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:07:18.473277 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:07:18.473659 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:07:18.473663 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:07:18.473665 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:07:18.473908 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:07:18.474042 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0809 02:07:18.474426 (+ 384us) server_negotiation.cc:299] Negotiation successful
0809 02:07:18.474555 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 02:07:18.477604 26097 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4a64508bda0746f1930f1fc924b1484c [term 1 FOLLOWER]: Refusing update from remote peer bfee98e4126949e1b107df21aba96eac: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:07:18.478580 26161 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bfee98e4126949e1b107df21aba96eac [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a64508bda0746f1930f1fc924b1484c" member_type: VOTER last_known_addr { host: "127.1.192.252" port: 37197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:07:18.479413 26016 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 86b298e244bc417f961a6eef09e72f6a [term 1 FOLLOWER]: Refusing update from remote peer bfee98e4126949e1b107df21aba96eac: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:07:18.483632 26164 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a64508bda0746f1930f1fc924b1484c [sys.catalog]: SysCatalogTable state changed. Reason: New leader bfee98e4126949e1b107df21aba96eac. Latest consensus state: current_term: 1 leader_uuid: "bfee98e4126949e1b107df21aba96eac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfee98e4126949e1b107df21aba96eac" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 44345 } } peers { permanent_uuid: "86b298e244bc417f961a6eef09e72f6a" member_type: VOTER last_known_addr { host: "127.1.192.253" port: 32939 } } peers { permanent_uuid: "4a64508bda0746f1930f1fc924b1484c" member_type: VOTER last_known_addr { host: "127.1.192.252" port: 37197 } } }
I20260809 02:07:18.483816 26164 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a64508bda0746f1930f1fc924b1484c [sys.catalog]: This master's current role is: FOLLOWER
I20260809 02:07:18.483987 26161 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bfee98e4126949e1b107df21aba96eac [LEADER]: Connected to new peer: Peer: permanent_uuid: "86b298e244bc417f961a6eef09e72f6a" member_type: VOTER last_known_addr { host: "127.1.192.253" port: 32939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:07:18.487911 26161 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfee98e4126949e1b107df21aba96eac [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bfee98e4126949e1b107df21aba96eac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfee98e4126949e1b107df21aba96eac" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 44345 } } peers { permanent_uuid: "86b298e244bc417f961a6eef09e72f6a" member_type: VOTER last_known_addr { host: "127.1.192.253" port: 32939 } } peers { permanent_uuid: "4a64508bda0746f1930f1fc924b1484c" member_type: VOTER last_known_addr { host: "127.1.192.252" port: 37197 } } }
I20260809 02:07:18.488092 26161 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfee98e4126949e1b107df21aba96eac [sys.catalog]: This master's current role is: LEADER
I20260809 02:07:18.488243 26185 mvcc.cc:205] Tried to move back new op lower bound from 7316444112796581888 to 7316444112660230144. Current Snapshot: MvccSnapshot[applied={T|T < 7316444112796581888}]
I20260809 02:07:18.489316 26139 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86b298e244bc417f961a6eef09e72f6a [sys.catalog]: SysCatalogTable state changed. Reason: New leader bfee98e4126949e1b107df21aba96eac. Latest consensus state: current_term: 1 leader_uuid: "bfee98e4126949e1b107df21aba96eac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfee98e4126949e1b107df21aba96eac" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 44345 } } peers { permanent_uuid: "86b298e244bc417f961a6eef09e72f6a" member_type: VOTER last_known_addr { host: "127.1.192.253" port: 32939 } } peers { permanent_uuid: "4a64508bda0746f1930f1fc924b1484c" member_type: VOTER last_known_addr { host: "127.1.192.252" port: 37197 } } }
I20260809 02:07:18.489468 26139 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86b298e244bc417f961a6eef09e72f6a [sys.catalog]: This master's current role is: FOLLOWER
I20260809 02:07:18.489816 26164 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a64508bda0746f1930f1fc924b1484c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bfee98e4126949e1b107df21aba96eac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfee98e4126949e1b107df21aba96eac" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 44345 } } peers { permanent_uuid: "86b298e244bc417f961a6eef09e72f6a" member_type: VOTER last_known_addr { host: "127.1.192.253" port: 32939 } } peers { permanent_uuid: "4a64508bda0746f1930f1fc924b1484c" member_type: VOTER last_known_addr { host: "127.1.192.252" port: 37197 } } }
I20260809 02:07:18.489949 26164 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a64508bda0746f1930f1fc924b1484c [sys.catalog]: This master's current role is: FOLLOWER
I20260809 02:07:18.490204 26186 mvcc.cc:205] Tried to move back new op lower bound from 7316444112796581888 to 7316444112660230144. Current Snapshot: MvccSnapshot[applied={T|T < 7316444112796581888}]
I20260809 02:07:18.493955 26163 catalog_manager.cc:1347] Generated new cluster ID: aac5427f07614fada33cea37befaeb08
I20260809 02:07:18.494063 26163 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:07:18.498657 26133 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfee98e4126949e1b107df21aba96eac [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bfee98e4126949e1b107df21aba96eac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfee98e4126949e1b107df21aba96eac" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 44345 } } peers { permanent_uuid: "86b298e244bc417f961a6eef09e72f6a" member_type: VOTER last_known_addr { host: "127.1.192.253" port: 32939 } } peers { permanent_uuid: "4a64508bda0746f1930f1fc924b1484c" member_type: VOTER last_known_addr { host: "127.1.192.252" port: 37197 } } }
I20260809 02:07:18.498831 26133 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfee98e4126949e1b107df21aba96eac [sys.catalog]: This master's current role is: LEADER
I20260809 02:07:18.500609 26139 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86b298e244bc417f961a6eef09e72f6a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bfee98e4126949e1b107df21aba96eac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfee98e4126949e1b107df21aba96eac" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 44345 } } peers { permanent_uuid: "86b298e244bc417f961a6eef09e72f6a" member_type: VOTER last_known_addr { host: "127.1.192.253" port: 32939 } } peers { permanent_uuid: "4a64508bda0746f1930f1fc924b1484c" member_type: VOTER last_known_addr { host: "127.1.192.252" port: 37197 } } }
I20260809 02:07:18.500765 26139 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86b298e244bc417f961a6eef09e72f6a [sys.catalog]: This master's current role is: FOLLOWER
I20260809 02:07:18.501379 26188 mvcc.cc:205] Tried to move back new op lower bound from 7316444112796581888 to 7316444112660230144. Current Snapshot: MvccSnapshot[applied={T|T < 7316444112796581888}]
I20260809 02:07:18.529116 26163 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:07:18.530534 26163 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:07:18.548188 26163 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bfee98e4126949e1b107df21aba96eac: Generated new TSK 0
I20260809 02:07:18.549259 26163 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 02:07:18.585433 26050 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:07:18.581610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48880 (local address 127.1.192.253:32939)
0809 02:07:18.581747 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0809 02:07:18.581752 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:07:18.583755 (+ 2003us) server_negotiation.cc:407] Connection header received
0809 02:07:18.584340 (+ 585us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:07:18.584345 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:07:18.584402 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:07:18.584458 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:07:18.585281 (+ 823us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48880: BlockingRecv error: recv got EOF from 127.0.0.1:48880 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260809 02:07:18.586944 25969 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:07:18.582692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43948 (local address 127.1.192.254:44345)
0809 02:07:18.582823 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0809 02:07:18.582828 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:07:18.585989 (+ 3161us) server_negotiation.cc:407] Connection header received
0809 02:07:18.586183 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:07:18.586186 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:07:18.586239 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:07:18.586291 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:07:18.586812 (+ 521us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43948: BlockingRecv error: recv got EOF from 127.0.0.1:43948 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260809 02:07:18.588573 26128 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:07:18.583383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43470 (local address 127.1.192.252:37197)
0809 02:07:18.583500 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0809 02:07:18.583504 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:07:18.587499 (+ 3995us) server_negotiation.cc:407] Connection header received
0809 02:07:18.587777 (+ 278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:07:18.587780 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:07:18.587831 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:07:18.587884 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:07:18.588433 (+ 549us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43470: BlockingRecv error: recv got EOF from 127.0.0.1:43470 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260809 02:07:18.594022 26050 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:07:18.591326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48892 (local address 127.1.192.253:32939)
0809 02:07:18.591462 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0809 02:07:18.591466 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:07:18.592820 (+ 1354us) server_negotiation.cc:407] Connection header received
0809 02:07:18.593125 (+ 305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:07:18.593128 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:07:18.593221 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:07:18.593280 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:07:18.593890 (+ 610us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48892: BlockingRecv error: recv got EOF from 127.0.0.1:48892 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260809 02:07:18.599685 26050 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:07:18.597970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48894 (local address 127.1.192.253:32939)
0809 02:07:18.598103 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0809 02:07:18.598108 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:07:18.598683 (+ 575us) server_negotiation.cc:407] Connection header received
0809 02:07:18.598884 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:07:18.598888 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:07:18.598940 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:07:18.598993 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:07:18.599560 (+ 567us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48894: BlockingRecv error: recv got EOF from 127.0.0.1:48894 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260809 02:07:18.607057 26050 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:07:18.604028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48896 (local address 127.1.192.253:32939)
0809 02:07:18.604157 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0809 02:07:18.604161 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:07:18.605995 (+ 1834us) server_negotiation.cc:407] Connection header received
0809 02:07:18.606223 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:07:18.606227 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:07:18.606282 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:07:18.606337 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:07:18.606934 (+ 597us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48896: BlockingRecv error: recv got EOF from 127.0.0.1:48896 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260809 02:07:18.615418 26050 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:07:18.613627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48900 (local address 127.1.192.253:32939)
0809 02:07:18.613780 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0809 02:07:18.613785 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:07:18.614401 (+ 616us) server_negotiation.cc:407] Connection header received
0809 02:07:18.614604 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:07:18.614608 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:07:18.614663 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:07:18.614718 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:07:18.615301 (+ 583us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48900: BlockingRecv error: recv got EOF from 127.0.0.1:48900 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
W20260809 02:07:18.628862 26050 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:07:18.626573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48916 (local address 127.1.192.253:32939)
0809 02:07:18.626726 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0809 02:07:18.626732 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:07:18.627563 (+ 831us) server_negotiation.cc:407] Connection header received
0809 02:07:18.627839 (+ 276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:07:18.627844 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:07:18.627918 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:07:18.627999 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:07:18.628723 (+ 724us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48916: BlockingRecv error: recv got EOF from 127.0.0.1:48916 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
W20260809 02:07:18.651084 26050 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:07:18.648539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48920 (local address 127.1.192.253:32939)
0809 02:07:18.648704 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0809 02:07:18.648710 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:07:18.649657 (+ 947us) server_negotiation.cc:407] Connection header received
0809 02:07:18.650009 (+ 352us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:07:18.650014 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:07:18.650087 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:07:18.650162 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:07:18.650915 (+ 753us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48920: BlockingRecv error: recv got EOF from 127.0.0.1:48920 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
W20260809 02:07:18.687911 26050 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:07:18.686171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48932 (local address 127.1.192.253:32939)
0809 02:07:18.686344 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0809 02:07:18.686349 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:07:18.686578 (+ 229us) server_negotiation.cc:407] Connection header received
0809 02:07:18.686965 (+ 387us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:07:18.686969 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:07:18.687024 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:07:18.687106 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:07:18.687779 (+ 673us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48932: BlockingRecv error: recv got EOF from 127.0.0.1:48932 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
W20260809 02:07:18.755844 26050 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:07:18.754486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48938 (local address 127.1.192.253:32939)
0809 02:07:18.754745 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0809 02:07:18.754749 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:07:18.754767 (+ 18us) server_negotiation.cc:407] Connection header received
0809 02:07:18.754909 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:07:18.754911 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:07:18.754960 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:07:18.755053 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:07:18.755715 (+ 662us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48938: BlockingRecv error: recv got EOF from 127.0.0.1:48938 (error 108)
Metrics: {"server-negotiator.queue_time_us":153}
W20260809 02:07:18.888051 26050 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:07:18.886674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48944 (local address 127.1.192.253:32939)
0809 02:07:18.886856 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 02:07:18.886861 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:07:18.886985 (+ 124us) server_negotiation.cc:407] Connection header received
0809 02:07:18.887215 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:07:18.887222 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:07:18.887273 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:07:18.887383 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:07:18.887899 (+ 516us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48944: BlockingRecv error: recv got EOF from 127.0.0.1:48944 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
W20260809 02:07:19.148095 26050 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:07:19.146577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48952 (local address 127.1.192.253:32939)
0809 02:07:19.146813 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0809 02:07:19.146823 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 02:07:19.147003 (+ 180us) server_negotiation.cc:407] Connection header received
0809 02:07:19.147168 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:07:19.147171 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:07:19.147224 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:07:19.147310 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:07:19.147971 (+ 661us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48952: BlockingRecv error: recv got EOF from 127.0.0.1:48952 (error 108)
Metrics: {"server-negotiator.queue_time_us":129}
I20260809 02:07:19.429693 26156 catalog_manager.cc:1259] Loaded cluster ID: aac5427f07614fada33cea37befaeb08
I20260809 02:07:19.429813 26156 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 86b298e244bc417f961a6eef09e72f6a: loading cluster ID for follower catalog manager: success
I20260809 02:07:19.431569 26156 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 86b298e244bc417f961a6eef09e72f6a: acquiring CA information for follower catalog manager: success
I20260809 02:07:19.432770 26156 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 86b298e244bc417f961a6eef09e72f6a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 02:07:19.457768 26180 catalog_manager.cc:1259] Loaded cluster ID: aac5427f07614fada33cea37befaeb08
I20260809 02:07:19.457859 26180 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 4a64508bda0746f1930f1fc924b1484c: loading cluster ID for follower catalog manager: success
I20260809 02:07:19.459589 26180 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 4a64508bda0746f1930f1fc924b1484c: acquiring CA information for follower catalog manager: success
I20260809 02:07:19.460875 26180 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 4a64508bda0746f1930f1fc924b1484c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 02:07:19.666848 26259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:07:19.662404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48962 (local address 127.1.192.253:32939)
0809 02:07:19.662750 (+ 346us) server_negotiation.cc:206] Beginning negotiation
0809 02:07:19.662754 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:07:19.662842 (+ 88us) server_negotiation.cc:407] Connection header received
0809 02:07:19.662975 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:07:19.662978 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:07:19.663119 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:07:19.663244 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:07:19.663584 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:07:19.664120 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:07:19.665069 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:07:19.665296 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:07:19.665383 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:07:19.665912 (+ 529us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:07:19.666360 (+ 448us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:07:19.666465 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0809 02:07:19.666576 (+ 111us) server_negotiation.cc:299] Negotiation successful
0809 02:07:19.666632 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":245,"thread_start_us":147,"threads_started":1}
I20260809 02:07:19.674923 26265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:07:19.670529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43484 (local address 127.1.192.252:37197)
0809 02:07:19.670857 (+ 328us) server_negotiation.cc:206] Beginning negotiation
0809 02:07:19.670861 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:07:19.670875 (+ 14us) server_negotiation.cc:407] Connection header received
0809 02:07:19.670969 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:07:19.670971 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:07:19.671102 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:07:19.671227 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:07:19.671475 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:07:19.672061 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:07:19.673094 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:07:19.673331 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:07:19.673422 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:07:19.673936 (+ 514us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:07:19.674402 (+ 466us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:07:19.674504 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 02:07:19.674658 (+ 154us) server_negotiation.cc:299] Negotiation successful
0809 02:07:19.674718 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":263,"thread_start_us":186,"threads_started":1}
I20260809 02:07:19.683131 26259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:07:19.678877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48976 (local address 127.1.192.253:32939)
0809 02:07:19.679032 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0809 02:07:19.679051 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0809 02:07:19.679300 (+ 249us) server_negotiation.cc:407] Connection header received
0809 02:07:19.679514 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:07:19.679517 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:07:19.679573 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:07:19.679623 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:07:19.680012 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:07:19.680627 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:07:19.681631 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:07:19.682033 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:07:19.682164 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:07:19.682470 (+ 306us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:07:19.682585 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:07:19.682698 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 02:07:19.682848 (+ 150us) server_negotiation.cc:299] Negotiation successful
0809 02:07:19.682916 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 02:07:19.683521 26265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:07:19.679390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43500 (local address 127.1.192.252:37197)
0809 02:07:19.679552 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0809 02:07:19.679560 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 02:07:19.679580 (+ 20us) server_negotiation.cc:407] Connection header received
0809 02:07:19.679745 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:07:19.679750 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:07:19.679810 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:07:19.679917 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:07:19.680255 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:07:19.681154 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:07:19.682231 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:07:19.682589 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:07:19.682663 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:07:19.683024 (+ 361us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:07:19.683109 (+ 85us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:07:19.683156 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0809 02:07:19.683324 (+ 168us) server_negotiation.cc:299] Negotiation successful
0809 02:07:19.683398 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 02:07:19.684412 26271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:07:19.678705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43958 (local address 127.1.192.254:44345)
0809 02:07:19.679435 (+ 730us) server_negotiation.cc:206] Beginning negotiation
0809 02:07:19.679440 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:07:19.679455 (+ 15us) server_negotiation.cc:407] Connection header received
0809 02:07:19.679571 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:07:19.679574 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:07:19.679750 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:07:19.679877 (+ 127us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:07:19.680214 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:07:19.681154 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:07:19.682571 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:07:19.682915 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:07:19.683014 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:07:19.683462 (+ 448us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:07:19.683856 (+ 394us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:07:19.683955 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 02:07:19.684230 (+ 275us) server_negotiation.cc:299] Negotiation successful
0809 02:07:19.684297 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":639,"thread_start_us":115,"threads_started":1}
I20260809 02:07:19.688256 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 25890
I20260809 02:07:19.697547 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 25972
I20260809 02:07:19.705653 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 26052
2026-08-09T02:07:19Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3280 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786240941968412-1795-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:07:19 dist-test-slave-ss4n krb5kdc[26277](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:07:19 dist-test-slave-ss4n krb5kdc[26277](info): set up 2 sockets
Aug 09 02:07:19 dist-test-slave-ss4n krb5kdc[26277](info): commencing operation
krb5kdc: starting...
W20260809 02:07:20.900885 1795 mini_kdc.cc:121] Time spent starting KDC: real 1.172s user 0.006s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786240941968412-1795-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786240941968412-1795-0/krb5kdc/test-user.keytab.
Aug 09 02:07:20 dist-test-slave-ss4n krb5kdc[26277](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241240, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:07:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:07:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.192.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.192.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.192.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab.
Entry for principal kudu/127.1.192.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab.
I20260809 02:07:21.045872 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786240941968412-1795-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786240941968412-1795-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786240941968412-1795-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.192.254:45251
--webserver_interface=127.1.192.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab
--principal=kudu/127.1.192.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:36679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.192.254:45251,127.1.192.253:33941,127.1.192.252:45071
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:07:21.163836 26291 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 02:07:21.164247 26291 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:07:21.164332 26291 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:07:21.164415 26291 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:07:21.168656 26291 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:07:21.168783 26291 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:07:21.168817 26291 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:07:21.168849 26291 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:07:21.168885 26291 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:07:21.174635 26291 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:36679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786240941968412-1795-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786240941968412-1795-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.192.254:45251,127.1.192.253:33941,127.1.192.252:45071
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.254.keytab
--principal=kudu/127.1.192.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.192.254:45251
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.192.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26291
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786240941968412-1795-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:07:21.175953 26291 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:07:21.177085 26291 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 02:07:21.186380 26291 server_base.cc:1034] running on GCE node
W20260809 02:07:21.186405 26297 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:07:21.186336 26296 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:07:21.186326 26299 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:07:21.187251 26291 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:07:21.188452 26291 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:07:21.189641 26291 hybrid_clock.cc:648] HybridClock initialized: now 1786241241189622 us; error 34 us; skew 500 ppm
Aug 09 02:07:21 dist-test-slave-ss4n krb5kdc[26277](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241241, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:07:21.192639 26291 init.cc:377] Logged in from keytab as kudu/127.1.192.254@KRBTEST.COM (short username kudu)
I20260809 02:07:21.193840 26291 webserver.cc:460] Webserver started at http://127.1.192.254:46191/ using document root <none> and password file <none>
I20260809 02:07:21.194500 26291 fs_manager.cc:359] Metadata directory not provided
I20260809 02:07:21.194607 26291 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:07:21.194875 26291 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:07:21.197264 26291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786240941968412-1795-0/minicluster-data/master-0/data/instance:
uuid: "58ee5ec92c3d4c7ba0675d5f8b1e7273"
format_stamp: "Formatted at 2026-08-09 02:07:21 on dist-test-slave-ss4n"
server_key: "78ec9456070c48375763bdc8f465b2d6"
server_key_iv: "7ddd58d9721c6577e200a8872aaec472"
server_key_version: "encryptionkey@0"
I20260809 02:07:21.198030 26291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786240941968412-1795-0/minicluster-data/master-0/wal/instance:
uuid: "58ee5ec92c3d4c7ba0675d5f8b1e7273"
format_stamp: "Formatted at 2026-08-09 02:07:21 on dist-test-slave-ss4n"
server_key: "78ec9456070c48375763bdc8f465b2d6"
server_key_iv: "7ddd58d9721c6577e200a8872aaec472"
server_key_version: "encryptionkey@0"
I20260809 02:07:21.202481 26291 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 02:07:21.205432 26307 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:07:21.207104 26291 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 02:07:21.207290 26291 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786240941968412-1795-0/minicluster-data/master-0/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786240941968412-1795-0/minicluster-data/master-0/wal
uuid: "58ee5ec92c3d4c7ba0675d5f8b1e7273"
format_stamp: "Formatted at 2026-08-09 02:07:21 on dist-test-slave-ss4n"
server_key: "78ec9456070c48375763bdc8f465b2d6"
server_key_iv: "7ddd58d9721c6577e200a8872aaec472"
server_key_version: "encryptionkey@0"
I20260809 02:07:21.207432 26291 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786240941968412-1795-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786240941968412-1795-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786240941968412-1795-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:07:21.229036 26291 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:07:21.232731 26291 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:07:21.233116 26291 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:07:21.241372 26291 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.254:45251
I20260809 02:07:21.241751 26364 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.254:45251 every 8 connection(s)
I20260809 02:07:21.242489 26291 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786240941968412-1795-0/minicluster-data/master-0/data/info.pb
I20260809 02:07:21.244439 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 26291
I20260809 02:07:21.244680 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786240941968412-1795-0/minicluster-data/master-0/wal/instance
I20260809 02:07:21.245066 1795 external_mini_cluster.cc:1428] Setting key 52c6be7c2d26621d7d4997e2de4f98fc
I20260809 02:07:21.245934 26365 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:07:21.250365 26365 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.192.254" port: 45251 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.1.192.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.192.253@KRBTEST.COM" created.
Aug 09 02:07:21 dist-test-slave-ss4n krb5kdc[26277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241241, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.254@KRBTEST.COM for kudu/127.1.192.254@KRBTEST.COM
I20260809 02:07:21.269846 26369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:07:21.253184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40568 (local address 127.1.192.254:45251)
0809 02:07:21.253898 (+ 714us) server_negotiation.cc:206] Beginning negotiation
0809 02:07:21.253918 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0809 02:07:21.254141 (+ 223us) server_negotiation.cc:407] Connection header received
0809 02:07:21.255714 (+ 1573us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:07:21.255720 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:07:21.255923 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:07:21.256099 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:07:21.257346 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:07:21.258623 (+ 1277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:07:21.259913 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:07:21.260424 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:07:21.260649 (+ 225us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:07:21.263657 (+ 3008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:07:21.263663 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:07:21.263675 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:07:21.263698 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:07:21.265797 (+ 2099us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:07:21.266551 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:07:21.266557 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:07:21.266558 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:07:21.266733 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:07:21.267187 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:07:21.267189 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:07:21.267191 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:07:21.267678 (+ 487us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:07:21.267790 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 02:07:21.268389 (+ 599us) server_negotiation.cc:299] Negotiation successful
0809 02:07:21.268688 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":527,"spinlock_wait_cycles":58240,"thread_start_us":221,"threads_started":1}
I20260809 02:07:21.269850 26368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:07:21.253184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:45251 (local address 127.0.0.1:40568)
0809 02:07:21.253931 (+ 747us) negotiation.cc:107] Waiting for socket to connect
0809 02:07:21.253999 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0809 02:07:21.255356 (+ 1357us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:07:21.256189 (+ 833us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:07:21.256199 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:07:21.256735 (+ 536us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:07:21.257178 (+ 443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:07:21.257197 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:07:21.258749 (+ 1552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:07:21.258752 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:07:21.259750 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:07:21.259766 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:07:21.260610 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:07:21.260613 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:07:21.260763 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:07:21.261554 (+ 791us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:07:21.261591 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:07:21.263418 (+ 1827us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:07:21.265940 (+ 2522us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:07:21.265952 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:07:21.265969 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:07:21.266397 (+ 428us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:07:21.266854 (+ 457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:07:21.266860 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:07:21.266864 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:07:21.267064 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:07:21.267811 (+ 747us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:07:21.267821 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:07:21.268034 (+ 213us) client_negotiation.cc:769] Sending connection context
0809 02:07:21.268275 (+ 241us) client_negotiation.cc:240] Negotiation successful
0809 02:07:21.268815 (+ 540us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":447,"mutex_wait_us":100,"thread_start_us":198,"threads_started":1}
I20260809 02:07:21.272382 26365 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.192.253" port: 33941 } has no permanent_uuid. Determining permanent_uuid...
I20260809 02:07:21.273440 26368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:07:21.272980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.253:33941 (local address 127.0.0.1:48680)
0809 02:07:21.273239 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0809 02:07:21.273346 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.253:33941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113}
W20260809 02:07:21.273813 26309 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.192.253:33941: connect: Connection refused (error 111)
I20260809 02:07:21.274924 26368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:07:21.274697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.253:33941 (local address 127.0.0.1:48690)
0809 02:07:21.274815 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0809 02:07:21.274866 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.253:33941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260809 02:07:21.275123 26365 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.192.253:33941: Network error: Client connection negotiation failed: client connection to 127.1.192.253:33941: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.192.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.253.keytab.
Entry for principal kudu/127.1.192.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.253.keytab.
I20260809 02:07:21.281875 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786240941968412-1795-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786240941968412-1795-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786240941968412-1795-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786240941968412-1795-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.192.253:33941
--webserver_interface=127.1.192.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.253.keytab
--principal=kudu/127.1.192.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:36679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.192.254:45251,127.1.192.253:33941,127.1.192.252:45071
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 02:07:21.324479 26365 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.192.253" port: 33941 } attempt: 1
I20260809 02:07:21.325655 26368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:07:21.325224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.253:33941 (local address 127.0.0.1:48704)
0809 02:07:21.325414 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0809 02:07:21.325507 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.253:33941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260809 02:07:21.326501 26368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:07:21.326270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.253:33941 (local address 127.0.0.1:48716)
0809 02:07:21.326385 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0809 02:07:21.326446 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.253:33941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260809 02:07:21.326637 26365 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.192.253:33941: Network error: Client connection negotiation failed: client connection to 127.1.192.253:33941: connect: Connection refused (error 111)
I20260809 02:07:21.394825 26365 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.192.253" port: 33941 } attempt: 2
I20260809 02:07:21.395819 26368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:07:21.395482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.253:33941 (local address 127.0.0.1:48720)
0809 02:07:21.395670 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0809 02:07:21.395737 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.253:33941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260809 02:07:21.396673 26368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:07:21.396398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.253:33941 (local address 127.0.0.1:48730)
0809 02:07:21.396541 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0809 02:07:21.396600 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.253:33941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260809 02:07:21.396938 26365 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.192.253:33941: Network error: Client connection negotiation failed: client connection to 127.1.192.253:33941: connect: Connection refused (error 111)
W20260809 02:07:21.401472 26372 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 02:07:21.401904 26372 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:07:21.401974 26372 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:07:21.402065 26372 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:07:21.406317 26372 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:07:21.406409 26372 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:07:21.406446 26372 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:07:21.406478 26372 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:07:21.406510 26372 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:07:21.412509 26372 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:36679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786240941968412-1795-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786240941968412-1795-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.1.192.254:45251,127.1.192.253:33941,127.1.192.252:45071
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.253.keytab
--principal=kudu/127.1.192.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.192.253:33941
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786240941968412-1795-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.1.192.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26372
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786240941968412-1795-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:07:21.413802 26372 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:07:21.414849 26372 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:07:21.424587 26381 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:07:21.424559 26377 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:07:21.424546 26378 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:07:21.425205 26372 server_base.cc:1034] running on GCE node
I20260809 02:07:21.425860 26372 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:07:21.427060 26372 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:07:21.428264 26372 hybrid_clock.cc:648] HybridClock initialized: now 1786241241428251 us; error 33 us; skew 500 ppm
Aug 09 02:07:21 dist-test-slave-ss4n krb5kdc[26277](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241241, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:07:21.436671 26372 init.cc:377] Logged in from keytab as kudu/127.1.192.253@KRBTEST.COM (short username kudu)
I20260809 02:07:21.437935 26372 webserver.cc:460] Webserver started at http://127.1.192.253:35181/ using document root <none> and password file <none>
I20260809 02:07:21.438531 26372 fs_manager.cc:359] Metadata directory not provided
I20260809 02:07:21.438597 26372 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:07:21.438789 26372 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:07:21.441049 26372 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786240941968412-1795-0/minicluster-data/master-1/data/instance:
uuid: "dbdf70fdfb9f45e89b9391a41dac6295"
format_stamp: "Formatted at 2026-08-09 02:07:21 on dist-test-slave-ss4n"
server_key: "37e2b307021c365d38f005ad67fc38c1"
server_key_iv: "85d02ddb2c4ad3e1eb60c2b973598778"
server_key_version: "encryptionkey@0"
I20260809 02:07:21.441700 26372 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786240941968412-1795-0/minicluster-data/master-1/wal/instance:
uuid: "dbdf70fdfb9f45e89b9391a41dac6295"
format_stamp: "Formatted at 2026-08-09 02:07:21 on dist-test-slave-ss4n"
server_key: "37e2b307021c365d38f005ad67fc38c1"
server_key_iv: "85d02ddb2c4ad3e1eb60c2b973598778"
server_key_version: "encryptionkey@0"
I20260809 02:07:21.445667 26372 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 02:07:21.448307 26388 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:07:21.449734 26372 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 02:07:21.449844 26372 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786240941968412-1795-0/minicluster-data/master-1/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786240941968412-1795-0/minicluster-data/master-1/wal
uuid: "dbdf70fdfb9f45e89b9391a41dac6295"
format_stamp: "Formatted at 2026-08-09 02:07:21 on dist-test-slave-ss4n"
server_key: "37e2b307021c365d38f005ad67fc38c1"
server_key_iv: "85d02ddb2c4ad3e1eb60c2b973598778"
server_key_version: "encryptionkey@0"
I20260809 02:07:21.449930 26372 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786240941968412-1795-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786240941968412-1795-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786240941968412-1795-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:07:21.467897 26372 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:07:21.471777 26372 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:07:21.471962 26372 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:07:21.479821 26372 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.253:33941
I20260809 02:07:21.480204 26442 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.253:33941 every 8 connection(s)
I20260809 02:07:21.480976 26372 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786240941968412-1795-0/minicluster-data/master-1/data/info.pb
I20260809 02:07:21.484020 26443 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:07:21.488135 26365 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.192.253" port: 33941 } attempt: 3
I20260809 02:07:21.487939 26443 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.192.254" port: 45251 } has no permanent_uuid. Determining permanent_uuid...
I20260809 02:07:21.490648 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 26372
I20260809 02:07:21.490777 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786240941968412-1795-0/minicluster-data/master-1/wal/instance
I20260809 02:07:21.491071 1795 external_mini_cluster.cc:1428] Setting key 1dc8992d28361c7712da2f874dd612eb
Aug 09 02:07:21 dist-test-slave-ss4n krb5kdc[26277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241241, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.254@KRBTEST.COM for kudu/127.1.192.253@KRBTEST.COM
Aug 09 02:07:21 dist-test-slave-ss4n krb5kdc[26277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241241, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.253@KRBTEST.COM for kudu/127.1.192.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.192.252@KRBTEST.COM; defaulting to no policy
I20260809 02:07:21.502655 26368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:07:21.488750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.253:33941 (local address 127.0.0.1:48742)
0809 02:07:21.488928 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0809 02:07:21.488947 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 02:07:21.489051 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:07:21.491650 (+ 2599us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:07:21.491656 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:07:21.491880 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:07:21.492042 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:07:21.492053 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:07:21.493794 (+ 1741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:07:21.493799 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:07:21.494316 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:07:21.494324 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:07:21.495136 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:07:21.495143 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:07:21.495311 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:07:21.496019 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:07:21.496040 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:07:21.497679 (+ 1639us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:07:21.500625 (+ 2946us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:07:21.500631 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:07:21.500637 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:07:21.500918 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:07:21.501300 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:07:21.501305 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:07:21.501307 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:07:21.501390 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:07:21.502054 (+ 664us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:07:21.502057 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:07:21.502193 (+ 136us) client_negotiation.cc:769] Sending connection context
0809 02:07:21.502306 (+ 113us) client_negotiation.cc:240] Negotiation successful
0809 02:07:21.502451 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67,"mutex_wait_us":26}
I20260809 02:07:21.503613 26369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:07:21.490006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40578 (local address 127.1.192.254:45251)
0809 02:07:21.490125 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0809 02:07:21.490130 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:07:21.490574 (+ 444us) server_negotiation.cc:407] Connection header received
0809 02:07:21.491382 (+ 808us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:07:21.491386 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:07:21.491445 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:07:21.491546 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:07:21.492612 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:07:21.493204 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:07:21.494856 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:07:21.495202 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:07:21.495338 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:07:21.498584 (+ 3246us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:07:21.498590 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:07:21.498593 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:07:21.498620 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:07:21.500288 (+ 1668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:07:21.500788 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:07:21.500793 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:07:21.500795 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:07:21.500881 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:07:21.501232 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:07:21.501236 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:07:21.501238 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:07:21.501534 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:07:21.501629 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 02:07:21.503336 (+ 1707us) server_negotiation.cc:299] Negotiation successful
0809 02:07:21.503470 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":50,"server-negotiator.queue_time_us":36,"spinlock_wait_cycles":99712}
I20260809 02:07:21.503659 26447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:07:21.489986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:45251 (local address 127.0.0.1:40578)
0809 02:07:21.490436 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0809 02:07:21.490486 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 02:07:21.491150 (+ 664us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:07:21.491612 (+ 462us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:07:21.491622 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:07:21.492006 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:07:21.492465 (+ 459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:07:21.492476 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:07:21.493357 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:07:21.493362 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:07:21.494724 (+ 1362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:07:21.494738 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:07:21.495514 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:07:21.495521 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:07:21.495687 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:07:21.496229 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:07:21.496245 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:07:21.498377 (+ 2132us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:07:21.500414 (+ 2037us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:07:21.500426 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:07:21.500438 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:07:21.500685 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:07:21.500977 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:07:21.500981 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:07:21.500983 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:07:21.501135 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:07:21.501634 (+ 499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:07:21.501642 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:07:21.501946 (+ 304us) client_negotiation.cc:769] Sending connection context
0809 02:07:21.502196 (+ 250us) client_negotiation.cc:240] Negotiation successful
0809 02:07:21.502614 (+ 418us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"mutex_wait_us":32,"thread_start_us":184,"threads_started":1}
I20260809 02:07:21.503665 26445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:07:21.489571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48742 (local address 127.1.192.253:33941)
0809 02:07:21.490043 (+ 472us) server_negotiation.cc:206] Beginning negotiation
0809 02:07:21.490063 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0809 02:07:21.490113 (+ 50us) server_negotiation.cc:407] Connection header received
0809 02:07:21.491123 (+ 1010us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:07:21.491148 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:07:21.491431 (+ 283us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:07:21.492225 (+ 794us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:07:21.492372 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:07:21.493670 (+ 1298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:07:21.494463 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:07:21.494998 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:07:21.495216 (+ 218us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:07:21.497852 (+ 2636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:07:21.497863 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:07:21.497882 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:07:21.497924 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:07:21.500458 (+ 2534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:07:21.501071 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:07:21.501079 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:07:21.501083 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:07:21.501171 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:07:21.501532 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:07:21.501535 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:07:21.501537 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:07:21.501934 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:07:21.502040 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 02:07:21.502445 (+ 405us) server_negotiation.cc:299] Negotiation successful
0809 02:07:21.502736 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":108,"server-negotiator.queue_time_us":313,"thread_start_us":164,"threads_started":1}
I20260809 02:07:21.505142 26443 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.192.253" port: 33941 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.192.252@KRBTEST.COM" created.
I20260809 02:07:21.506275 26365 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.192.252" port: 45071 } has no permanent_uuid. Determining permanent_uuid...
I20260809 02:07:21.507099 26368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:07:21.506812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.252:45071 (local address 127.0.0.1:51798)
0809 02:07:21.506956 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0809 02:07:21.507019 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.252:45071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260809 02:07:21.508106 26368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:07:21.507755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.252:45071 (local address 127.0.0.1:51808)
0809 02:07:21.507966 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0809 02:07:21.508027 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.252:45071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134}
W20260809 02:07:21.508252 26365 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.192.252:45071: Network error: Client connection negotiation failed: client connection to 127.1.192.252:45071: connect: Connection refused (error 111)
Aug 09 02:07:21 dist-test-slave-ss4n krb5kdc[26277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241241, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.253@KRBTEST.COM for kudu/127.1.192.253@KRBTEST.COM
I20260809 02:07:21.514817 26447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:07:21.505755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.253:33941 (local address 127.0.0.1:48746)
0809 02:07:21.505885 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0809 02:07:21.505903 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 02:07:21.505994 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:07:21.506338 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:07:21.506340 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:07:21.506489 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:07:21.506563 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:07:21.506567 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:07:21.507480 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:07:21.507487 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:07:21.508340 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:07:21.508351 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:07:21.508965 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:07:21.508968 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:07:21.509042 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:07:21.509638 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:07:21.509658 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:07:21.511218 (+ 1560us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:07:21.513186 (+ 1968us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:07:21.513189 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:07:21.513191 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:07:21.513383 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:07:21.513762 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:07:21.513764 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:07:21.513767 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:07:21.513849 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:07:21.514231 (+ 382us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:07:21.514234 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:07:21.514316 (+ 82us) client_negotiation.cc:769] Sending connection context
0809 02:07:21.514430 (+ 114us) client_negotiation.cc:240] Negotiation successful
0809 02:07:21.514591 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260809 02:07:21.514817 26445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:07:21.505834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48746 (local address 127.1.192.253:33941)
0809 02:07:21.505938 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0809 02:07:21.505942 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:07:21.505957 (+ 15us) server_negotiation.cc:407] Connection header received
0809 02:07:21.506096 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:07:21.506102 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:07:21.506161 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:07:21.506241 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:07:21.506677 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:07:21.507377 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:07:21.508452 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:07:21.508849 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:07:21.508967 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:07:21.511383 (+ 2416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:07:21.511386 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:07:21.511388 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:07:21.511430 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:07:21.513057 (+ 1627us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:07:21.513569 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:07:21.513575 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:07:21.513578 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:07:21.513666 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:07:21.513943 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:07:21.513946 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:07:21.513948 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:07:21.514141 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:07:21.514243 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 02:07:21.514486 (+ 243us) server_negotiation.cc:299] Negotiation successful
0809 02:07:21.514650 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":64,"server-negotiator.queue_time_us":29}
I20260809 02:07:21.515781 26443 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.192.252" port: 45071 } has no permanent_uuid. Determining permanent_uuid...
I20260809 02:07:21.516542 26447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:07:21.516272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.252:45071 (local address 127.0.0.1:51822)
0809 02:07:21.516387 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0809 02:07:21.516470 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.252:45071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260809 02:07:21.516906 26392 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.192.252:45071: connect: Connection refused (error 111)
I20260809 02:07:21.518023 26447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:07:21.517781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.252:45071 (local address 127.0.0.1:51838)
0809 02:07:21.517925 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0809 02:07:21.517969 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.252:45071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
W20260809 02:07:21.518280 26443 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.192.252:45071: Network error: Client connection negotiation failed: client connection to 127.1.192.252:45071: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.192.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.252.keytab.
Entry for principal kudu/127.1.192.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.252.keytab.
I20260809 02:07:21.525198 1795 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
/tmp/dist-test-taskRWTcX9/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786240941968412-1795-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786240941968412-1795-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786240941968412-1795-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786240941968412-1795-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.192.252:45071
--webserver_interface=127.1.192.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.252.keytab
--principal=kudu/127.1.192.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.192.212:36679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.192.254:45251,127.1.192.253:33941,127.1.192.252:45071
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786240941968412-1795-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786240941968412-1795-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786240941968412-1795-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 02:07:21.539463 26365 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.192.252" port: 45071 } attempt: 1
I20260809 02:07:21.540434 26368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:07:21.540107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.252:45071 (local address 127.0.0.1:51854)
0809 02:07:21.540276 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0809 02:07:21.540352 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.252:45071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260809 02:07:21.541311 26368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:07:21.541061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.252:45071 (local address 127.0.0.1:51856)
0809 02:07:21.541198 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0809 02:07:21.541257 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.252:45071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260809 02:07:21.541455 26365 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.192.252:45071: Network error: Client connection negotiation failed: client connection to 127.1.192.252:45071: connect: Connection refused (error 111)
I20260809 02:07:21.567546 26443 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.192.252" port: 45071 } attempt: 1
I20260809 02:07:21.568504 26447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:07:21.568197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.252:45071 (local address 127.0.0.1:51866)
0809 02:07:21.568360 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0809 02:07:21.568427 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.252:45071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260809 02:07:21.569304 26447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:07:21.569076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.252:45071 (local address 127.0.0.1:51868)
0809 02:07:21.569191 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0809 02:07:21.569246 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.252:45071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260809 02:07:21.569439 26443 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.192.252:45071: Network error: Client connection negotiation failed: client connection to 127.1.192.252:45071: connect: Connection refused (error 111)
I20260809 02:07:21.616706 26365 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.192.252" port: 45071 } attempt: 2
I20260809 02:07:21.617727 26368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:07:21.617340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.252:45071 (local address 127.0.0.1:51874)
0809 02:07:21.617550 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0809 02:07:21.617631 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.252:45071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260809 02:07:21.618654 26368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:07:21.618415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.252:45071 (local address 127.0.0.1:51884)
0809 02:07:21.618528 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0809 02:07:21.618596 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.252:45071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260809 02:07:21.618844 26365 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.192.252:45071: Network error: Client connection negotiation failed: client connection to 127.1.192.252:45071: connect: Connection refused (error 111)
I20260809 02:07:21.637626 26443 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.192.252" port: 45071 } attempt: 2
I20260809 02:07:21.638505 26447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:07:21.638212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.252:45071 (local address 127.0.0.1:51892)
0809 02:07:21.638359 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0809 02:07:21.638430 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.252:45071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260809 02:07:21.639509 26447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:07:21.639226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.252:45071 (local address 127.0.0.1:51904)
0809 02:07:21.639387 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0809 02:07:21.639435 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.192.252:45071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
W20260809 02:07:21.639683 26443 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.192.252:45071: Network error: Client connection negotiation failed: client connection to 127.1.192.252:45071: connect: Connection refused (error 111)
W20260809 02:07:21.644371 26452 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 02:07:21.644706 26452 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:07:21.644731 26452 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:07:21.644773 26452 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:07:21.648991 26452 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:07:21.649070 26452 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:07:21.649091 26452 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:07:21.649109 26452 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:07:21.649125 26452 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:07:21.654870 26452 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.192.212:36679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786240941968412-1795-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786240941968412-1795-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.1.192.254:45251,127.1.192.253:33941,127.1.192.252:45071
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786240941968412-1795-0/krb5kdc/kudu_127.1.192.252.keytab
--principal=kudu/127.1.192.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.192.252:45071
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786240941968412-1795-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.1.192.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26452
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786240941968412-1795-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 01:21:56 UTC on ccycloud.quasar-xrrvkm.root.comops.site
build id 4575763
I20260809 02:07:21.656117 26452 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:07:21.657140 26452 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:07:21.666412 26459 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:07:21.666514 26458 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:07:21.666432 26461 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:07:21.666844 26452 server_base.cc:1034] running on GCE node
I20260809 02:07:21.667454 26452 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:07:21.668721 26452 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:07:21.669942 26452 hybrid_clock.cc:648] HybridClock initialized: now 1786241241669926 us; error 38 us; skew 500 ppm
Aug 09 02:07:21 dist-test-slave-ss4n krb5kdc[26277](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241241, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:07:21.673425 26452 init.cc:377] Logged in from keytab as kudu/127.1.192.252@KRBTEST.COM (short username kudu)
I20260809 02:07:21.674698 26452 webserver.cc:460] Webserver started at http://127.1.192.252:38337/ using document root <none> and password file <none>
I20260809 02:07:21.675351 26452 fs_manager.cc:359] Metadata directory not provided
I20260809 02:07:21.675457 26452 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:07:21.675724 26452 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:07:21.678120 26452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786240941968412-1795-0/minicluster-data/master-2/data/instance:
uuid: "2b983118cd5e4514a494e9f0c317a637"
format_stamp: "Formatted at 2026-08-09 02:07:21 on dist-test-slave-ss4n"
server_key: "363b9a5707e56515927126ba72d8f302"
server_key_iv: "530fd344c4c9ed54cfb7e31a90e978db"
server_key_version: "encryptionkey@0"
I20260809 02:07:21.678781 26452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786240941968412-1795-0/minicluster-data/master-2/wal/instance:
uuid: "2b983118cd5e4514a494e9f0c317a637"
format_stamp: "Formatted at 2026-08-09 02:07:21 on dist-test-slave-ss4n"
server_key: "363b9a5707e56515927126ba72d8f302"
server_key_iv: "530fd344c4c9ed54cfb7e31a90e978db"
server_key_version: "encryptionkey@0"
I20260809 02:07:21.683147 26452 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 02:07:21.685864 26468 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:07:21.687285 26452 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 02:07:21.687467 26452 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786240941968412-1795-0/minicluster-data/master-2/data,/tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786240941968412-1795-0/minicluster-data/master-2/wal
uuid: "2b983118cd5e4514a494e9f0c317a637"
format_stamp: "Formatted at 2026-08-09 02:07:21 on dist-test-slave-ss4n"
server_key: "363b9a5707e56515927126ba72d8f302"
server_key_iv: "530fd344c4c9ed54cfb7e31a90e978db"
server_key_version: "encryptionkey@0"
I20260809 02:07:21.687600 26452 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786240941968412-1795-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786240941968412-1795-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786240941968412-1795-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:07:21.704587 26452 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:07:21.708518 26452 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:07:21.708806 26452 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:07:21.717082 26452 rpc_server.cc:307] RPC server started. Bound to: 127.1.192.252:45071
I20260809 02:07:21.717460 26522 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.192.252:45071 every 8 connection(s)
I20260809 02:07:21.718032 26365 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.192.252" port: 45071 } attempt: 3
I20260809 02:07:21.718322 26452 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786240941968412-1795-0/minicluster-data/master-2/data/info.pb
I20260809 02:07:21.721267 26523 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:07:21.723539 1795 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu as pid 26452
I20260809 02:07:21.723722 1795 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRWTcX9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786240941968412-1795-0/minicluster-data/master-2/wal/instance
I20260809 02:07:21.724053 1795 external_mini_cluster.cc:1428] Setting key 1c11b07d2dcf4f3fb85b0c9058f2d928
I20260809 02:07:21.725950 26523 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.192.254" port: 45251 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 02:07:21 dist-test-slave-ss4n krb5kdc[26277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241241, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.254@KRBTEST.COM for kudu/127.1.192.252@KRBTEST.COM
I20260809 02:07:21.730852 26443 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.192.252" port: 45071 } attempt: 3
Aug 09 02:07:21 dist-test-slave-ss4n krb5kdc[26277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241240, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.192.252@KRBTEST.COM
I20260809 02:07:21.734032 26368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:07:21.718636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.252:45071 (local address 127.0.0.1:51906)
0809 02:07:21.718842 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0809 02:07:21.718864 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 02:07:21.718991 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:07:21.721511 (+ 2520us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:07:21.721515 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:07:21.721756 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:07:21.721883 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:07:21.721892 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:07:21.723525 (+ 1633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:07:21.723529 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:07:21.724362 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:07:21.724377 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:07:21.725410 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:07:21.725417 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:07:21.725560 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:07:21.726240 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:07:21.726256 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:07:21.727933 (+ 1677us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:07:21.730719 (+ 2786us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:07:21.730725 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:07:21.730727 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:07:21.731043 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:07:21.731508 (+ 465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:07:21.731515 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:07:21.731519 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:07:21.731619 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:07:21.732463 (+ 844us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:07:21.732467 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:07:21.732578 (+ 111us) client_negotiation.cc:769] Sending connection context
0809 02:07:21.732687 (+ 109us) client_negotiation.cc:240] Negotiation successful
0809 02:07:21.733394 (+ 707us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":71,"mutex_wait_us":519}
I20260809 02:07:21.734459 26525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:07:21.719457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51906 (local address 127.1.192.252:45071)
0809 02:07:21.719922 (+ 465us) server_negotiation.cc:206] Beginning negotiation
0809 02:07:21.719950 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0809 02:07:21.719995 (+ 45us) server_negotiation.cc:407] Connection header received
0809 02:07:21.720720 (+ 725us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:07:21.720738 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:07:21.721139 (+ 401us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:07:21.721549 (+ 410us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:07:21.722014 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:07:21.723161 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:07:21.724526 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:07:21.725229 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:07:21.725444 (+ 215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:07:21.728154 (+ 2710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:07:21.728159 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:07:21.728179 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:07:21.728208 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:07:21.730593 (+ 2385us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:07:21.731191 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:07:21.731199 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:07:21.731208 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:07:21.731371 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:07:21.731762 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:07:21.731767 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:07:21.731768 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:07:21.732330 (+ 562us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:07:21.732820 (+ 490us) server_negotiation.cc:1035] Waiting for connection context
0809 02:07:21.733000 (+ 180us) server_negotiation.cc:299] Negotiation successful
0809 02:07:21.733326 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":155,"threads_started":1}
Aug 09 02:07:21 dist-test-slave-ss4n krb5kdc[26277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241241, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.252@KRBTEST.COM for kudu/127.1.192.254@KRBTEST.COM
I20260809 02:07:21.738097 26526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:07:21.724730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51910 (local address 127.1.192.252:45071)
0809 02:07:21.725116 (+ 386us) server_negotiation.cc:206] Beginning negotiation
0809 02:07:21.725128 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 02:07:21.725304 (+ 176us) server_negotiation.cc:407] Connection header received
0809 02:07:21.725530 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:07:21.725533 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:07:21.725584 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:07:21.725685 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:07:21.726671 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:07:21.727377 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:07:21.728806 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:07:21.729156 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:07:21.729280 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:07:21.733226 (+ 3946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:07:21.733231 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:07:21.733233 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:07:21.733255 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:07:21.736156 (+ 2901us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:07:21.736736 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:07:21.736740 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:07:21.736741 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:07:21.736825 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:07:21.737176 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:07:21.737181 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:07:21.737186 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:07:21.737372 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:07:21.737603 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0809 02:07:21.737820 (+ 217us) server_negotiation.cc:299] Negotiation successful
0809 02:07:21.737931 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":302,"thread_start_us":116,"threads_started":1}
Aug 09 02:07:21 dist-test-slave-ss4n krb5kdc[26277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241241, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.253@KRBTEST.COM for kudu/127.1.192.252@KRBTEST.COM
I20260809 02:07:21.740350 26365 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 58ee5ec92c3d4c7ba0675d5f8b1e7273: Bootstrap starting.
I20260809 02:07:21.743355 26447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:07:21.731308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.252:45071 (local address 127.0.0.1:51922)
0809 02:07:21.731503 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0809 02:07:21.731518 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 02:07:21.731617 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:07:21.732397 (+ 780us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:07:21.732400 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:07:21.732630 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:07:21.732746 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:07:21.732754 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:07:21.734484 (+ 1730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:07:21.734487 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:07:21.735204 (+ 717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:07:21.735215 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:07:21.735865 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:07:21.735870 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:07:21.735982 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:07:21.736484 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:07:21.736504 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:07:21.738706 (+ 2202us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:07:21.741153 (+ 2447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:07:21.741157 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:07:21.741159 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:07:21.741413 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:07:21.741784 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:07:21.741787 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:07:21.741789 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:07:21.741872 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:07:21.742770 (+ 898us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:07:21.742773 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:07:21.742886 (+ 113us) client_negotiation.cc:769] Sending connection context
0809 02:07:21.742994 (+ 108us) client_negotiation.cc:240] Negotiation successful
0809 02:07:21.743174 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121}
I20260809 02:07:21.744400 26529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:07:21.731873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51922 (local address 127.1.192.252:45071)
0809 02:07:21.732140 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0809 02:07:21.732148 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 02:07:21.732162 (+ 14us) server_negotiation.cc:407] Connection header received
0809 02:07:21.732196 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:07:21.732200 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:07:21.732239 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:07:21.732922 (+ 683us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:07:21.733116 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:07:21.734069 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:07:21.735334 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:07:21.735707 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:07:21.736205 (+ 498us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:07:21.738860 (+ 2655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:07:21.738864 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:07:21.738868 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:07:21.738893 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:07:21.741010 (+ 2117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:07:21.741570 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:07:21.741573 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:07:21.741575 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:07:21.741661 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:07:21.741980 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:07:21.741983 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:07:21.741985 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:07:21.742634 (+ 649us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:07:21.743946 (+ 1312us) server_negotiation.cc:1035] Waiting for connection context
0809 02:07:21.744029 (+ 83us) server_negotiation.cc:299] Negotiation successful
0809 02:07:21.744163 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":208,"thread_start_us":112,"threads_started":1}
I20260809 02:07:21.744465 26369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:07:21.728967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40586 (local address 127.1.192.254:45251)
0809 02:07:21.729142 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0809 02:07:21.729166 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0809 02:07:21.729183 (+ 17us) server_negotiation.cc:407] Connection header received
0809 02:07:21.729225 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:07:21.729229 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:07:21.729276 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:07:21.729348 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:07:21.730380 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:07:21.731169 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:07:21.732545 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:07:21.732949 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:07:21.733118 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:07:21.738917 (+ 5799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:07:21.738923 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:07:21.738925 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:07:21.738955 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:07:21.740924 (+ 1969us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:07:21.741572 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:07:21.741576 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:07:21.741577 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:07:21.741661 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:07:21.741982 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:07:21.741985 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:07:21.741986 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:07:21.742298 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:07:21.742408 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 02:07:21.744136 (+ 1728us) server_negotiation.cc:299] Negotiation successful
0809 02:07:21.744283 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":65}
I20260809 02:07:21.744666 26365 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 58ee5ec92c3d4c7ba0675d5f8b1e7273: Neither blocks nor log segments found. Creating new log.
I20260809 02:07:21.745121 26528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:07:21.728199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.254:45251 (local address 127.0.0.1:40586)
0809 02:07:21.728635 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0809 02:07:21.728713 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0809 02:07:21.728848 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:07:21.729548 (+ 700us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:07:21.729556 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:07:21.730068 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:07:21.730223 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:07:21.730231 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:07:21.731313 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:07:21.731317 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:07:21.732405 (+ 1088us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:07:21.732414 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:07:21.735139 (+ 2725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:07:21.735146 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:07:21.735265 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:07:21.735878 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:07:21.735894 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:07:21.738750 (+ 2856us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:07:21.741069 (+ 2319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:07:21.741083 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:07:21.741090 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:07:21.741413 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:07:21.741783 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:07:21.741786 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:07:21.741788 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:07:21.741872 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:07:21.742431 (+ 559us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:07:21.742438 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:07:21.743862 (+ 1424us) client_negotiation.cc:769] Sending connection context
0809 02:07:21.744683 (+ 821us) client_negotiation.cc:240] Negotiation successful
0809 02:07:21.744928 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":147,"threads_started":1}
I20260809 02:07:21.746608 26523 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.192.253" port: 33941 } has no permanent_uuid. Determining permanent_uuid...
I20260809 02:07:21.748670 26443 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dbdf70fdfb9f45e89b9391a41dac6295: Bootstrap starting.
Aug 09 02:07:21 dist-test-slave-ss4n krb5kdc[26277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241240, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.192.254@KRBTEST.COM
Aug 09 02:07:21 dist-test-slave-ss4n krb5kdc[26277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241241, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.252@KRBTEST.COM for kudu/127.1.192.253@KRBTEST.COM
I20260809 02:07:21.752173 26443 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dbdf70fdfb9f45e89b9391a41dac6295: Neither blocks nor log segments found. Creating new log.
I20260809 02:07:21.753291 26443 log.cc:824] T 00000000000000000000000000000000 P dbdf70fdfb9f45e89b9391a41dac6295: Log is configured to *not* fsync() on all Append() calls
I20260809 02:07:21.753898 26365 log.cc:824] T 00000000000000000000000000000000 P 58ee5ec92c3d4c7ba0675d5f8b1e7273: Log is configured to *not* fsync() on all Append() calls
I20260809 02:07:21.755954 26530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:07:21.743039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40590 (local address 127.1.192.254:45251)
0809 02:07:21.743682 (+ 643us) server_negotiation.cc:206] Beginning negotiation
0809 02:07:21.743696 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 02:07:21.743710 (+ 14us) server_negotiation.cc:407] Connection header received
0809 02:07:21.743748 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:07:21.743751 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:07:21.743791 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:07:21.744712 (+ 921us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:07:21.744808 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:07:21.745586 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:07:21.746564 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:07:21.746885 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:07:21.747840 (+ 955us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:07:21.750872 (+ 3032us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:07:21.750876 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:07:21.750879 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:07:21.750909 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:07:21.752678 (+ 1769us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:07:21.753310 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:07:21.753316 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:07:21.753318 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:07:21.753521 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:07:21.754724 (+ 1203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:07:21.754731 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:07:21.754732 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:07:21.755007 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:07:21.755215 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0809 02:07:21.755576 (+ 361us) server_negotiation.cc:299] Negotiation successful
0809 02:07:21.755730 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":550,"thread_start_us":124,"threads_started":1}
I20260809 02:07:21.756292 26445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:07:21.747281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48754 (local address 127.1.192.253:33941)
0809 02:07:21.747435 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0809 02:07:21.747442 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 02:07:21.747460 (+ 18us) server_negotiation.cc:407] Connection header received
0809 02:07:21.747500 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:07:21.747503 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:07:21.747550 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:07:21.747644 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:07:21.748124 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:07:21.748618 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:07:21.749407 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:07:21.749773 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:07:21.749888 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:07:21.752483 (+ 2595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:07:21.752489 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:07:21.752493 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:07:21.752531 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:07:21.754273 (+ 1742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:07:21.754795 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:07:21.754799 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:07:21.754800 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:07:21.754883 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:07:21.755210 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:07:21.755214 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:07:21.755216 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:07:21.755483 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:07:21.755609 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0809 02:07:21.756005 (+ 396us) server_negotiation.cc:299] Negotiation successful
0809 02:07:21.756134 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 02:07:21.756672 26365 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 58ee5ec92c3d4c7ba0675d5f8b1e7273: No bootstrap required, opened a new log
I20260809 02:07:21.757077 26528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:07:21.747190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.253:33941 (local address 127.0.0.1:48754)
0809 02:07:21.747302 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0809 02:07:21.747318 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 02:07:21.747404 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:07:21.747692 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:07:21.747697 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:07:21.747884 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:07:21.748015 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:07:21.748022 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:07:21.748741 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:07:21.748746 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:07:21.749275 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:07:21.749284 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:07:21.749892 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:07:21.749896 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:07:21.749997 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:07:21.750532 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:07:21.750546 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:07:21.752315 (+ 1769us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:07:21.754394 (+ 2079us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:07:21.754398 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:07:21.754400 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:07:21.754676 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:07:21.755002 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:07:21.755004 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:07:21.755006 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:07:21.755093 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:07:21.755673 (+ 580us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:07:21.755676 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:07:21.755789 (+ 113us) client_negotiation.cc:769] Sending connection context
0809 02:07:21.756733 (+ 944us) client_negotiation.cc:240] Negotiation successful
0809 02:07:21.756903 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260809 02:07:21.758100 26523 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.192.252" port: 45071 } has no permanent_uuid. Determining permanent_uuid...
I20260809 02:07:21.761143 26365 raft_consensus.cc:348] T 00000000000000000000000000000000 P 58ee5ec92c3d4c7ba0675d5f8b1e7273 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58ee5ec92c3d4c7ba0675d5f8b1e7273" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 45251 } } peers { permanent_uuid: "dbdf70fdfb9f45e89b9391a41dac6295" member_type: VOTER last_known_addr { host: "127.1.192.253" port: 33941 } } peers { permanent_uuid: "2b983118cd5e4514a494e9f0c317a637" member_type: VOTER last_known_addr { host: "127.1.192.252" port: 45071 } }
I20260809 02:07:21.761427 26365 raft_consensus.cc:729] T 00000000000000000000000000000000 P 58ee5ec92c3d4c7ba0675d5f8b1e7273 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58ee5ec92c3d4c7ba0675d5f8b1e7273, State: Initialized, Role: FOLLOWER
I20260809 02:07:21.762447 26365 consensus_queue.cc:260] T 00000000000000000000000000000000 P 58ee5ec92c3d4c7ba0675d5f8b1e7273 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58ee5ec92c3d4c7ba0675d5f8b1e7273" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 45251 } } peers { permanent_uuid: "dbdf70fdfb9f45e89b9391a41dac6295" member_type: VOTER last_known_addr { host: "127.1.192.253" port: 33941 } } peers { permanent_uuid: "2b983118cd5e4514a494e9f0c317a637" member_type: VOTER last_known_addr { host: "127.1.192.252" port: 45071 } }
I20260809 02:07:21.763078 26533 sys_catalog.cc:455] T 00000000000000000000000000000000 P 58ee5ec92c3d4c7ba0675d5f8b1e7273 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58ee5ec92c3d4c7ba0675d5f8b1e7273" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 45251 } } peers { permanent_uuid: "dbdf70fdfb9f45e89b9391a41dac6295" member_type: VOTER last_known_addr { host: "127.1.192.253" port: 33941 } } peers { permanent_uuid: "2b983118cd5e4514a494e9f0c317a637" member_type: VOTER last_known_addr { host: "127.1.192.252" port: 45071 } } }
Aug 09 02:07:21 dist-test-slave-ss4n krb5kdc[26277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241240, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.192.253@KRBTEST.COM
I20260809 02:07:21.763321 26533 sys_catalog.cc:458] T 00000000000000000000000000000000 P 58ee5ec92c3d4c7ba0675d5f8b1e7273 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 02:07:21.764117 26365 sys_catalog.cc:564] T 00000000000000000000000000000000 P 58ee5ec92c3d4c7ba0675d5f8b1e7273 [sys.catalog]: configured and running, proceeding with master startup.
Aug 09 02:07:21 dist-test-slave-ss4n krb5kdc[26277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241241, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.192.252@KRBTEST.COM for kudu/127.1.192.252@KRBTEST.COM
I20260809 02:07:21.767676 26443 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dbdf70fdfb9f45e89b9391a41dac6295: No bootstrap required, opened a new log
I20260809 02:07:21.767740 26445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:07:21.758194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48756 (local address 127.1.192.253:33941)
0809 02:07:21.758301 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0809 02:07:21.758305 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:07:21.758317 (+ 12us) server_negotiation.cc:407] Connection header received
0809 02:07:21.758417 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:07:21.758420 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:07:21.758477 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:07:21.758568 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:07:21.759307 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:07:21.759839 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:07:21.760636 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:07:21.760861 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:07:21.760973 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:07:21.763824 (+ 2851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:07:21.763828 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:07:21.763831 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:07:21.763861 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:07:21.765841 (+ 1980us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:07:21.766310 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:07:21.766314 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:07:21.766316 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:07:21.766405 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:07:21.766729 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:07:21.766733 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:07:21.766735 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:07:21.766972 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:07:21.767163 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0809 02:07:21.767429 (+ 266us) server_negotiation.cc:299] Negotiation successful
0809 02:07:21.767555 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 02:07:21.771448 26529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:07:21.758648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51926 (local address 127.1.192.252:45071)
0809 02:07:21.761627 (+ 2979us) server_negotiation.cc:206] Beginning negotiation
0809 02:07:21.761631 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:07:21.761644 (+ 13us) server_negotiation.cc:407] Connection header received
0809 02:07:21.761683 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:07:21.761686 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:07:21.761738 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:07:21.761847 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:07:21.762341 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:07:21.763172 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:07:21.764234 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:07:21.764644 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:07:21.764777 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:07:21.767618 (+ 2841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:07:21.767622 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:07:21.767624 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:07:21.767647 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:07:21.769586 (+ 1939us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:07:21.770087 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:07:21.770091 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:07:21.770093 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:07:21.770169 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:07:21.770461 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:07:21.770464 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:07:21.770466 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:07:21.770715 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:07:21.770798 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 02:07:21.771113 (+ 315us) server_negotiation.cc:299] Negotiation successful
0809 02:07:21.771268 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2887}
I20260809 02:07:21.772243 26528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:07:21.758621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.192.252:45071 (local address 127.0.0.1:51926)
0809 02:07:21.758890 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0809 02:07:21.758904 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 02:07:21.758984 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:07:21.761858 (+ 2874us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:07:21.761861 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:07:21.762068 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:07:21.762161 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:07:21.762168 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:07:21.763304 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:07:21.763307 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:07:21.764084 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:07:21.764098 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:07:21.765089 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:07:21.765093 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:07:21.765209 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:07:21.765884 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:07:21.765904 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:07:21.767478 (+ 1574us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:07:21.769726 (+ 2248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:07:21.769730 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:07:21.769732 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:07:21.769987 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:07:21.770274 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:07:21.770277 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:07:21.770280 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:07:21.770362 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:07:21.770861 (+ 499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:07:21.770864 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:07:21.770951 (+ 87us) client_negotiation.cc:769] Sending connection context
0809 02:07:21.771912 (+ 961us) client_negotiation.cc:240] Negotiation successful
0809 02:07:21.772067 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189}
I20260809 02:07:21.773226 1795 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 02:07:21.773106 26443 raft_consensus.cc:348] T 00000000000000000000000000000000 P dbdf70fdfb9f45e89b9391a41dac6295 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58ee5ec92c3d4c7ba0675d5f8b1e7273" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 45251 } } peers { permanent_uuid: "dbdf70fdfb9f45e89b9391a41dac6295" member_type: VOTER last_known_addr { host: "127.1.192.253" port: 33941 } } peers { permanent_uuid: "2b983118cd5e4514a494e9f0c317a637" member_type: VOTER last_known_addr { host: "127.1.192.252" port: 45071 } }
I20260809 02:07:21.773401 26443 raft_consensus.cc:729] T 00000000000000000000000000000000 P dbdf70fdfb9f45e89b9391a41dac6295 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbdf70fdfb9f45e89b9391a41dac6295, State: Initialized, Role: FOLLOWER
I20260809 02:07:21.773809 26443 consensus_queue.cc:260] T 00000000000000000000000000000000 P dbdf70fdfb9f45e89b9391a41dac6295 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58ee5ec92c3d4c7ba0675d5f8b1e7273" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 45251 } } peers { permanent_uuid: "dbdf70fdfb9f45e89b9391a41dac6295" member_type: VOTER last_known_addr { host: "127.1.192.253" port: 33941 } } peers { permanent_uuid: "2b983118cd5e4514a494e9f0c317a637" member_type: VOTER last_known_addr { host: "127.1.192.252" port: 45071 } }
I20260809 02:07:21.774308 26545 sys_catalog.cc:455] T 00000000000000000000000000000000 P dbdf70fdfb9f45e89b9391a41dac6295 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58ee5ec92c3d4c7ba0675d5f8b1e7273" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 45251 } } peers { permanent_uuid: "dbdf70fdfb9f45e89b9391a41dac6295" member_type: VOTER last_known_addr { host: "127.1.192.253" port: 33941 } } peers { permanent_uuid: "2b983118cd5e4514a494e9f0c317a637" member_type: VOTER last_known_addr { host: "127.1.192.252" port: 45071 } } }
I20260809 02:07:21.774449 26545 sys_catalog.cc:458] T 00000000000000000000000000000000 P dbdf70fdfb9f45e89b9391a41dac6295 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 02:07:21.775138 26335 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "58ee5ec92c3d4c7ba0675d5f8b1e7273"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40590
I20260809 02:07:21.775404 26335 raft_consensus.cc:482] T 00000000000000000000000000000000 P 58ee5ec92c3d4c7ba0675d5f8b1e7273 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 02:07:21.775445 26443 sys_catalog.cc:564] T 00000000000000000000000000000000 P dbdf70fdfb9f45e89b9391a41dac6295 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:07:21.776418 26523 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b983118cd5e4514a494e9f0c317a637: Bootstrap starting.
W20260809 02:07:21.779435 26544 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 58ee5ec92c3d4c7ba0675d5f8b1e7273: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260809 02:07:21.779505 26523 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2b983118cd5e4514a494e9f0c317a637: Neither blocks nor log segments found. Creating new log.
W20260809 02:07:21.779747 26544 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 02:07:21.775496 26291 master_runner.cc:186] Error getting master registration for 127.1.192.253:33941: OK, instance_id { permanent_uuid: "dbdf70fdfb9f45e89b9391a41dac6295" instance_seqno: 1786241241477688 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 02:07:21.780455 26291 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 02:07:21.780467 26523 log.cc:824] T 00000000000000000000000000000000 P 2b983118cd5e4514a494e9f0c317a637: Log is configured to *not* fsync() on all Append() calls
I20260809 02:07:21.775516 26335 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 58ee5ec92c3d4c7ba0675d5f8b1e7273 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:07:21.782212 26335 raft_consensus.cc:504] T 00000000000000000000000000000000 P 58ee5ec92c3d4c7ba0675d5f8b1e7273 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58ee5ec92c3d4c7ba0675d5f8b1e7273" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 45251 } } peers { permanent_uuid: "dbdf70fdfb9f45e89b9391a41dac6295" member_type: VOTER last_known_addr { host: "127.1.192.253" port: 33941 } } peers { permanent_uuid: "2b983118cd5e4514a494e9f0c317a637" member_type: VOTER last_known_addr { host: "127.1.192.252" port: 45071 } }
I20260809 02:07:21.782696 26523 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b983118cd5e4514a494e9f0c317a637: No bootstrap required, opened a new log
I20260809 02:07:21.784070 26415 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "58ee5ec92c3d4c7ba0675d5f8b1e7273" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "dbdf70fdfb9f45e89b9391a41dac6295"
I20260809 02:07:21.784334 26415 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dbdf70fdfb9f45e89b9391a41dac6295 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:07:21.785863 26415 raft_consensus.cc:2454] T 00000000000000000000000000000000 P dbdf70fdfb9f45e89b9391a41dac6295 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58ee5ec92c3d4c7ba0675d5f8b1e7273 in term 1.
I20260809 02:07:21.786541 26309 leader_election.cc:302] T 00000000000000000000000000000000 P 58ee5ec92c3d4c7ba0675d5f8b1e7273 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58ee5ec92c3d4c7ba0675d5f8b1e7273, dbdf70fdfb9f45e89b9391a41dac6295; no voters:
I20260809 02:07:21.786911 26533 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 58ee5ec92c3d4c7ba0675d5f8b1e7273 [term 1 FOLLOWER]: Leader election won for term 1
W20260809 02:07:21.787012 26557 catalog_manager.cc:1558] T 00000000000000000000000000000000 P dbdf70fdfb9f45e89b9391a41dac6295: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 02:07:21.787086 26557 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 02:07:21.787160 26533 raft_consensus.cc:686] T 00000000000000000000000000000000 P 58ee5ec92c3d4c7ba0675d5f8b1e7273 [term 1 LEADER]: Becoming Leader. State: Replica: 58ee5ec92c3d4c7ba0675d5f8b1e7273, State: Running, Role: LEADER
I20260809 02:07:21.787529 26533 consensus_queue.cc:237] T 00000000000000000000000000000000 P 58ee5ec92c3d4c7ba0675d5f8b1e7273 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58ee5ec92c3d4c7ba0675d5f8b1e7273" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 45251 } } peers { permanent_uuid: "dbdf70fdfb9f45e89b9391a41dac6295" member_type: VOTER last_known_addr { host: "127.1.192.253" port: 33941 } } peers { permanent_uuid: "2b983118cd5e4514a494e9f0c317a637" member_type: VOTER last_known_addr { host: "127.1.192.252" port: 45071 } }
I20260809 02:07:21.787734 26523 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2b983118cd5e4514a494e9f0c317a637 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58ee5ec92c3d4c7ba0675d5f8b1e7273" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 45251 } } peers { permanent_uuid: "dbdf70fdfb9f45e89b9391a41dac6295" member_type: VOTER last_known_addr { host: "127.1.192.253" port: 33941 } } peers { permanent_uuid: "2b983118cd5e4514a494e9f0c317a637" member_type: VOTER last_known_addr { host: "127.1.192.252" port: 45071 } }
I20260809 02:07:21.787974 26523 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2b983118cd5e4514a494e9f0c317a637 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b983118cd5e4514a494e9f0c317a637, State: Initialized, Role: FOLLOWER
I20260809 02:07:21.788305 26523 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2b983118cd5e4514a494e9f0c317a637 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58ee5ec92c3d4c7ba0675d5f8b1e7273" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 45251 } } peers { permanent_uuid: "dbdf70fdfb9f45e89b9391a41dac6295" member_type: VOTER last_known_addr { host: "127.1.192.253" port: 33941 } } peers { permanent_uuid: "2b983118cd5e4514a494e9f0c317a637" member_type: VOTER last_known_addr { host: "127.1.192.252" port: 45071 } }
I20260809 02:07:21.789745 26533 sys_catalog.cc:455] T 00000000000000000000000000000000 P 58ee5ec92c3d4c7ba0675d5f8b1e7273 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 58ee5ec92c3d4c7ba0675d5f8b1e7273. Latest consensus state: current_term: 1 leader_uuid: "58ee5ec92c3d4c7ba0675d5f8b1e7273" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58ee5ec92c3d4c7ba0675d5f8b1e7273" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 45251 } } peers { permanent_uuid: "dbdf70fdfb9f45e89b9391a41dac6295" member_type: VOTER last_known_addr { host: "127.1.192.253" port: 33941 } } peers { permanent_uuid: "2b983118cd5e4514a494e9f0c317a637" member_type: VOTER last_known_addr { host: "127.1.192.252" port: 45071 } } }
I20260809 02:07:21.789867 26533 sys_catalog.cc:458] T 00000000000000000000000000000000 P 58ee5ec92c3d4c7ba0675d5f8b1e7273 [sys.catalog]: This master's current role is: LEADER
I20260809 02:07:21.790002 26523 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2b983118cd5e4514a494e9f0c317a637 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:07:21.791165 26496 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "58ee5ec92c3d4c7ba0675d5f8b1e7273" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "2b983118cd5e4514a494e9f0c317a637"
I20260809 02:07:21.791453 26496 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2b983118cd5e4514a494e9f0c317a637 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:07:21.787775 26372 master_runner.cc:186] Error getting master registration for 127.1.192.252:45071: OK, instance_id { permanent_uuid: "2b983118cd5e4514a494e9f0c317a637" instance_seqno: 1786241241714913 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 02:07:21.791999 26372 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 02:07:21.793043 26496 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2b983118cd5e4514a494e9f0c317a637 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58ee5ec92c3d4c7ba0675d5f8b1e7273 in term 1.
I20260809 02:07:21.793830 26335 leader_election.cc:288] T 00000000000000000000000000000000 P 58ee5ec92c3d4c7ba0675d5f8b1e7273 [CANDIDATE]: Term 1 election: Requested vote from peers dbdf70fdfb9f45e89b9391a41dac6295 (127.1.192.253:33941), 2b983118cd5e4514a494e9f0c317a637 (127.1.192.252:45071)
W20260809 02:07:21.797358 26574 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2b983118cd5e4514a494e9f0c317a637: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 02:07:21.797459 26574 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 02:07:21.799486 26563 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:07:21.800460 26563 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:07:21.808786 26561 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b983118cd5e4514a494e9f0c317a637 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58ee5ec92c3d4c7ba0675d5f8b1e7273" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 45251 } } peers { permanent_uuid: "dbdf70fdfb9f45e89b9391a41dac6295" member_type: VOTER last_known_addr { host: "127.1.192.253" port: 33941 } } peers { permanent_uuid: "2b983118cd5e4514a494e9f0c317a637" member_type: VOTER last_known_addr { host: "127.1.192.252" port: 45071 } } }
I20260809 02:07:21.808976 26561 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b983118cd5e4514a494e9f0c317a637 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 02:07:21.819394 26445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:07:21.796284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48766 (local address 127.1.192.253:33941)
0809 02:07:21.796415 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0809 02:07:21.796419 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:07:21.802392 (+ 5973us) server_negotiation.cc:407] Connection header received
0809 02:07:21.802618 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:07:21.802621 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:07:21.802677 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:07:21.802734 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:07:21.803862 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:07:21.804620 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:07:21.806132 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:07:21.806491 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:07:21.806577 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:07:21.811821 (+ 5244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:07:21.811827 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:07:21.811833 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:07:21.811871 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:07:21.814105 (+ 2234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:07:21.816235 (+ 2130us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:07:21.816240 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:07:21.816242 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:07:21.816362 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:07:21.816732 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:07:21.816737 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:07:21.816739 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:07:21.816982 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:07:21.817119 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0809 02:07:21.819049 (+ 1930us) server_negotiation.cc:299] Negotiation successful
0809 02:07:21.819210 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 02:07:21.827100 26496 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2b983118cd5e4514a494e9f0c317a637 [term 1 FOLLOWER]: Refusing update from remote peer 58ee5ec92c3d4c7ba0675d5f8b1e7273: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:07:21.828128 26560 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 58ee5ec92c3d4c7ba0675d5f8b1e7273 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b983118cd5e4514a494e9f0c317a637" member_type: VOTER last_known_addr { host: "127.1.192.252" port: 45071 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:07:21.829521 26530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:07:21.801954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40594 (local address 127.1.192.254:45251)
0809 02:07:21.802088 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0809 02:07:21.802093 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:07:21.805774 (+ 3681us) server_negotiation.cc:407] Connection header received
0809 02:07:21.805898 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:07:21.805901 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:07:21.805952 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:07:21.806029 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:07:21.809230 (+ 3201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:07:21.810108 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:07:21.814994 (+ 4886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:07:21.815426 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:07:21.815516 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:07:21.823070 (+ 7554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:07:21.823076 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:07:21.823079 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:07:21.823117 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:07:21.825074 (+ 1957us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:07:21.827910 (+ 2836us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:07:21.827915 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:07:21.827917 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:07:21.828051 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:07:21.828399 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:07:21.828403 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:07:21.828405 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:07:21.828647 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:07:21.828782 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0809 02:07:21.829163 (+ 381us) server_negotiation.cc:299] Negotiation successful
0809 02:07:21.829324 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 02:07:21.832855 26561 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b983118cd5e4514a494e9f0c317a637 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 58ee5ec92c3d4c7ba0675d5f8b1e7273. Latest consensus state: current_term: 1 leader_uuid: "58ee5ec92c3d4c7ba0675d5f8b1e7273" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58ee5ec92c3d4c7ba0675d5f8b1e7273" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 45251 } } peers { permanent_uuid: "dbdf70fdfb9f45e89b9391a41dac6295" member_type: VOTER last_known_addr { host: "127.1.192.253" port: 33941 } } peers { permanent_uuid: "2b983118cd5e4514a494e9f0c317a637" member_type: VOTER last_known_addr { host: "127.1.192.252" port: 45071 } } }
I20260809 02:07:21.833009 26561 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b983118cd5e4514a494e9f0c317a637 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 02:07:21.834249 26415 raft_consensus.cc:1261] T 00000000000000000000000000000000 P dbdf70fdfb9f45e89b9391a41dac6295 [term 1 FOLLOWER]: Refusing update from remote peer 58ee5ec92c3d4c7ba0675d5f8b1e7273: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:07:21.834940 26560 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 58ee5ec92c3d4c7ba0675d5f8b1e7273 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dbdf70fdfb9f45e89b9391a41dac6295" member_type: VOTER last_known_addr { host: "127.1.192.253" port: 33941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:07:21.837754 26529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:07:21.802427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51934 (local address 127.1.192.252:45071)
0809 02:07:21.802535 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0809 02:07:21.802540 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:07:21.817894 (+ 15354us) server_negotiation.cc:407] Connection header received
0809 02:07:21.821857 (+ 3963us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:07:21.821861 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:07:21.821925 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:07:21.821983 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:07:21.825742 (+ 3759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:07:21.826605 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:07:21.831423 (+ 4818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:07:21.831791 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:07:21.831874 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:07:21.833919 (+ 2045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:07:21.833924 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:07:21.833927 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:07:21.833958 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:07:21.835747 (+ 1789us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:07:21.836292 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:07:21.836296 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:07:21.836298 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:07:21.836385 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:07:21.836671 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:07:21.836674 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:07:21.836676 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:07:21.836912 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:07:21.837044 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0809 02:07:21.837409 (+ 365us) server_negotiation.cc:299] Negotiation successful
0809 02:07:21.837590 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 02:07:21.839535 26545 sys_catalog.cc:455] T 00000000000000000000000000000000 P dbdf70fdfb9f45e89b9391a41dac6295 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 58ee5ec92c3d4c7ba0675d5f8b1e7273. Latest consensus state: current_term: 1 leader_uuid: "58ee5ec92c3d4c7ba0675d5f8b1e7273" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58ee5ec92c3d4c7ba0675d5f8b1e7273" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 45251 } } peers { permanent_uuid: "dbdf70fdfb9f45e89b9391a41dac6295" member_type: VOTER last_known_addr { host: "127.1.192.253" port: 33941 } } peers { permanent_uuid: "2b983118cd5e4514a494e9f0c317a637" member_type: VOTER last_known_addr { host: "127.1.192.252" port: 45071 } } }
I20260809 02:07:21.839666 26545 sys_catalog.cc:458] T 00000000000000000000000000000000 P dbdf70fdfb9f45e89b9391a41dac6295 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 02:07:21.842718 26560 sys_catalog.cc:455] T 00000000000000000000000000000000 P 58ee5ec92c3d4c7ba0675d5f8b1e7273 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "58ee5ec92c3d4c7ba0675d5f8b1e7273" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58ee5ec92c3d4c7ba0675d5f8b1e7273" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 45251 } } peers { permanent_uuid: "dbdf70fdfb9f45e89b9391a41dac6295" member_type: VOTER last_known_addr { host: "127.1.192.253" port: 33941 } } peers { permanent_uuid: "2b983118cd5e4514a494e9f0c317a637" member_type: VOTER last_known_addr { host: "127.1.192.252" port: 45071 } } }
I20260809 02:07:21.842849 26560 sys_catalog.cc:458] T 00000000000000000000000000000000 P 58ee5ec92c3d4c7ba0675d5f8b1e7273 [sys.catalog]: This master's current role is: LEADER
I20260809 02:07:21.845266 26545 sys_catalog.cc:455] T 00000000000000000000000000000000 P dbdf70fdfb9f45e89b9391a41dac6295 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "58ee5ec92c3d4c7ba0675d5f8b1e7273" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58ee5ec92c3d4c7ba0675d5f8b1e7273" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 45251 } } peers { permanent_uuid: "dbdf70fdfb9f45e89b9391a41dac6295" member_type: VOTER last_known_addr { host: "127.1.192.253" port: 33941 } } peers { permanent_uuid: "2b983118cd5e4514a494e9f0c317a637" member_type: VOTER last_known_addr { host: "127.1.192.252" port: 45071 } } }
I20260809 02:07:21.845383 26545 sys_catalog.cc:458] T 00000000000000000000000000000000 P dbdf70fdfb9f45e89b9391a41dac6295 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 02:07:21.846493 26533 sys_catalog.cc:455] T 00000000000000000000000000000000 P 58ee5ec92c3d4c7ba0675d5f8b1e7273 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "58ee5ec92c3d4c7ba0675d5f8b1e7273" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58ee5ec92c3d4c7ba0675d5f8b1e7273" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 45251 } } peers { permanent_uuid: "dbdf70fdfb9f45e89b9391a41dac6295" member_type: VOTER last_known_addr { host: "127.1.192.253" port: 33941 } } peers { permanent_uuid: "2b983118cd5e4514a494e9f0c317a637" member_type: VOTER last_known_addr { host: "127.1.192.252" port: 45071 } } }
I20260809 02:07:21.846621 26533 sys_catalog.cc:458] T 00000000000000000000000000000000 P 58ee5ec92c3d4c7ba0675d5f8b1e7273 [sys.catalog]: This master's current role is: LEADER
I20260809 02:07:21.847082 26588 mvcc.cc:205] Tried to move back new op lower bound from 7316444126471593984 to 7316444126365704192. Current Snapshot: MvccSnapshot[applied={T|T < 7316444126471593984}]
I20260809 02:07:21.847812 26561 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b983118cd5e4514a494e9f0c317a637 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "58ee5ec92c3d4c7ba0675d5f8b1e7273" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58ee5ec92c3d4c7ba0675d5f8b1e7273" member_type: VOTER last_known_addr { host: "127.1.192.254" port: 45251 } } peers { permanent_uuid: "dbdf70fdfb9f45e89b9391a41dac6295" member_type: VOTER last_known_addr { host: "127.1.192.253" port: 33941 } } peers { permanent_uuid: "2b983118cd5e4514a494e9f0c317a637" member_type: VOTER last_known_addr { host: "127.1.192.252" port: 45071 } } }
I20260809 02:07:21.847927 26561 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b983118cd5e4514a494e9f0c317a637 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 02:07:21.848730 26585 mvcc.cc:205] Tried to move back new op lower bound from 7316444126471593984 to 7316444126365704192. Current Snapshot: MvccSnapshot[applied={T|T < 7316444126471593984}]
I20260809 02:07:21.848928 26586 mvcc.cc:205] Tried to move back new op lower bound from 7316444126471593984 to 7316444126365704192. Current Snapshot: MvccSnapshot[applied={T|T < 7316444126471593984}]
I20260809 02:07:21.854235 26563 catalog_manager.cc:1347] Generated new cluster ID: bee99c693ab04ec68d572e4c19ce7669
I20260809 02:07:21.854312 26563 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:07:21.890960 26563 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:07:21.892735 26563 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:07:21.929044 26563 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 58ee5ec92c3d4c7ba0675d5f8b1e7273: Generated new TSK 0
I20260809 02:07:21.930261 26563 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 02:07:21.970139 26530 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:07:21.966131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49720 (local address 127.1.192.254:45251)
0809 02:07:21.966312 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 02:07:21.966317 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:07:21.968629 (+ 2312us) server_negotiation.cc:407] Connection header received
0809 02:07:21.968864 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:07:21.968868 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:07:21.968930 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:07:21.969006 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:07:21.969977 (+ 971us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49720: BlockingRecv error: recv got EOF from 127.0.0.1:49720 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260809 02:07:21.972113 26529 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:07:21.967028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47032 (local address 127.1.192.252:45071)
0809 02:07:21.967164 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0809 02:07:21.967169 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:07:21.971036 (+ 3867us) server_negotiation.cc:407] Connection header received
0809 02:07:21.971250 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:07:21.971254 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:07:21.971311 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:07:21.971368 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:07:21.971970 (+ 602us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47032: BlockingRecv error: recv got EOF from 127.0.0.1:47032 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260809 02:07:21.974054 26445 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:07:21.967892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46796 (local address 127.1.192.253:33941)
0809 02:07:21.968031 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0809 02:07:21.968036 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:07:21.972933 (+ 4897us) server_negotiation.cc:407] Connection header received
0809 02:07:21.973165 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:07:21.973169 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:07:21.973244 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:07:21.973304 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:07:21.973906 (+ 602us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46796: BlockingRecv error: recv got EOF from 127.0.0.1:46796 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
W20260809 02:07:21.980232 26445 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:07:21.977562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46802 (local address 127.1.192.253:33941)
0809 02:07:21.977697 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0809 02:07:21.977702 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:07:21.978902 (+ 1200us) server_negotiation.cc:407] Connection header received
0809 02:07:21.979219 (+ 317us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:07:21.979223 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:07:21.979322 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:07:21.979385 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:07:21.980086 (+ 701us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46802: BlockingRecv error: recv got EOF from 127.0.0.1:46802 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260809 02:07:21.987139 26445 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:07:21.985100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46806 (local address 127.1.192.253:33941)
0809 02:07:21.985234 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0809 02:07:21.985239 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:07:21.985923 (+ 684us) server_negotiation.cc:407] Connection header received
0809 02:07:21.986200 (+ 277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:07:21.986204 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:07:21.986274 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:07:21.986340 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:07:21.987002 (+ 662us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46806: BlockingRecv error: recv got EOF from 127.0.0.1:46806 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260809 02:07:21.994342 26445 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:07:21.992269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46820 (local address 127.1.192.253:33941)
0809 02:07:21.992408 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0809 02:07:21.992413 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:07:21.993092 (+ 679us) server_negotiation.cc:407] Connection header received
0809 02:07:21.993344 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:07:21.993348 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:07:21.993410 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:07:21.993476 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:07:21.994199 (+ 723us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46820: BlockingRecv error: recv got EOF from 127.0.0.1:46820 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
W20260809 02:07:22.003779 26445 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:07:22.001618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46836 (local address 127.1.192.253:33941)
0809 02:07:22.001813 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 02:07:22.001819 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:07:22.002576 (+ 757us) server_negotiation.cc:407] Connection header received
0809 02:07:22.002817 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:07:22.002822 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:07:22.002885 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:07:22.002966 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:07:22.003639 (+ 673us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46836: BlockingRecv error: recv got EOF from 127.0.0.1:46836 (error 108)
Metrics: {"server-negotiator.queue_time_us":101}
W20260809 02:07:22.017066 26445 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:07:22.015024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46848 (local address 127.1.192.253:33941)
0809 02:07:22.015154 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0809 02:07:22.015159 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:07:22.015813 (+ 654us) server_negotiation.cc:407] Connection header received
0809 02:07:22.016068 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:07:22.016073 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:07:22.016136 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:07:22.016189 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:07:22.016911 (+ 722us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46848: BlockingRecv error: recv got EOF from 127.0.0.1:46848 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260809 02:07:22.037668 26445 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:07:22.036147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46858 (local address 127.1.192.253:33941)
0809 02:07:22.036314 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0809 02:07:22.036319 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:07:22.036474 (+ 155us) server_negotiation.cc:407] Connection header received
0809 02:07:22.036752 (+ 278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:07:22.036756 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:07:22.036811 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:07:22.036921 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:07:22.037517 (+ 596us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46858: BlockingRecv error: recv got EOF from 127.0.0.1:46858 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
W20260809 02:07:22.073972 26445 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:07:22.072473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46864 (local address 127.1.192.253:33941)
0809 02:07:22.072681 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0809 02:07:22.072687 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:07:22.072844 (+ 157us) server_negotiation.cc:407] Connection header received
0809 02:07:22.073052 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:07:22.073056 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:07:22.073121 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:07:22.073229 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:07:22.073827 (+ 598us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46864: BlockingRecv error: recv got EOF from 127.0.0.1:46864 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
W20260809 02:07:22.141927 26445 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:07:22.140426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46868 (local address 127.1.192.253:33941)
0809 02:07:22.140647 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0809 02:07:22.140653 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:07:22.140855 (+ 202us) server_negotiation.cc:407] Connection header received
0809 02:07:22.141059 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:07:22.141062 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:07:22.141113 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:07:22.141214 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:07:22.141770 (+ 556us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46868: BlockingRecv error: recv got EOF from 127.0.0.1:46868 (error 108)
Metrics: {"server-negotiator.queue_time_us":107}
W20260809 02:07:22.273885 26445 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:07:22.272357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46884 (local address 127.1.192.253:33941)
0809 02:07:22.272497 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0809 02:07:22.272501 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:07:22.272598 (+ 97us) server_negotiation.cc:407] Connection header received
0809 02:07:22.272765 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:07:22.272768 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:07:22.272826 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:07:22.272914 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:07:22.273725 (+ 811us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46884: BlockingRecv error: recv got EOF from 127.0.0.1:46884 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
W20260809 02:07:22.533999 26445 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:07:22.532461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46890 (local address 127.1.192.253:33941)
0809 02:07:22.532704 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0809 02:07:22.532710 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:07:22.532854 (+ 144us) server_negotiation.cc:407] Connection header received
0809 02:07:22.533068 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:07:22.533072 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:07:22.533134 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:07:22.533282 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:07:22.533874 (+ 592us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46890: BlockingRecv error: recv got EOF from 127.0.0.1:46890 (error 108)
Metrics: {"server-negotiator.queue_time_us":144}
I20260809 02:07:22.788419 26557 catalog_manager.cc:1259] Loaded cluster ID: bee99c693ab04ec68d572e4c19ce7669
I20260809 02:07:22.788509 26557 catalog_manager.cc:1552] T 00000000000000000000000000000000 P dbdf70fdfb9f45e89b9391a41dac6295: loading cluster ID for follower catalog manager: success
I20260809 02:07:22.790421 26557 catalog_manager.cc:1574] T 00000000000000000000000000000000 P dbdf70fdfb9f45e89b9391a41dac6295: acquiring CA information for follower catalog manager: success
I20260809 02:07:22.791929 26557 catalog_manager.cc:1602] T 00000000000000000000000000000000 P dbdf70fdfb9f45e89b9391a41dac6295: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 02:07:22.798736 26574 catalog_manager.cc:1259] Loaded cluster ID: bee99c693ab04ec68d572e4c19ce7669
I20260809 02:07:22.798820 26574 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 2b983118cd5e4514a494e9f0c317a637: loading cluster ID for follower catalog manager: success
I20260809 02:07:22.800619 26574 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 2b983118cd5e4514a494e9f0c317a637: acquiring CA information for follower catalog manager: success
I20260809 02:07:22.801926 26574 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 2b983118cd5e4514a494e9f0c317a637: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 02:07:23.052691 26662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:07:23.048462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46906 (local address 127.1.192.253:33941)
0809 02:07:23.048731 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0809 02:07:23.048735 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:07:23.048751 (+ 16us) server_negotiation.cc:407] Connection header received
0809 02:07:23.048949 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:07:23.048953 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:07:23.049104 (+ 151us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:07:23.049218 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:07:23.049602 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:07:23.050188 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:07:23.051082 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:07:23.051332 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:07:23.051416 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:07:23.051742 (+ 326us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:07:23.052268 (+ 526us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:07:23.052368 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0809 02:07:23.052439 (+ 71us) server_negotiation.cc:299] Negotiation successful
0809 02:07:23.052491 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":201,"thread_start_us":79,"threads_started":1}
I20260809 02:07:23.060264 26668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:07:23.056010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47046 (local address 127.1.192.252:45071)
0809 02:07:23.056282 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0809 02:07:23.056286 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:07:23.056301 (+ 15us) server_negotiation.cc:407] Connection header received
0809 02:07:23.056460 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:07:23.056462 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:07:23.056582 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:07:23.056685 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:07:23.056944 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:07:23.057548 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:07:23.058512 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:07:23.058765 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:07:23.058858 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:07:23.059196 (+ 338us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:07:23.059622 (+ 426us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:07:23.059720 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0809 02:07:23.059933 (+ 213us) server_negotiation.cc:299] Negotiation successful
0809 02:07:23.060073 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":203,"thread_start_us":118,"threads_started":1}
I20260809 02:07:23.068953 26668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:07:23.064713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47056 (local address 127.1.192.252:45071)
0809 02:07:23.064870 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0809 02:07:23.064875 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:07:23.065127 (+ 252us) server_negotiation.cc:407] Connection header received
0809 02:07:23.065354 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:07:23.065358 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:07:23.065414 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:07:23.065559 (+ 145us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:07:23.065825 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:07:23.066548 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:07:23.067639 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:07:23.067921 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:07:23.068055 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:07:23.068325 (+ 270us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:07:23.068431 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:07:23.068507 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 02:07:23.068727 (+ 220us) server_negotiation.cc:299] Negotiation successful
0809 02:07:23.068799 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 02:07:23.069440 26662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:07:23.064077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46920 (local address 127.1.192.253:33941)
0809 02:07:23.064440 (+ 363us) server_negotiation.cc:206] Beginning negotiation
0809 02:07:23.064445 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:07:23.064461 (+ 16us) server_negotiation.cc:407] Connection header received
0809 02:07:23.064650 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:07:23.064658 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:07:23.064731 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:07:23.064789 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:07:23.065308 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:07:23.066224 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:07:23.067654 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:07:23.068060 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:07:23.068141 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:07:23.068628 (+ 487us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:07:23.068720 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:07:23.068792 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 02:07:23.069207 (+ 415us) server_negotiation.cc:299] Negotiation successful
0809 02:07:23.069286 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":248}
I20260809 02:07:23.070015 26674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:07:23.064152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49736 (local address 127.1.192.254:45251)
0809 02:07:23.064638 (+ 486us) server_negotiation.cc:206] Beginning negotiation
0809 02:07:23.064643 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:07:23.064902 (+ 259us) server_negotiation.cc:407] Connection header received
0809 02:07:23.065178 (+ 276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:07:23.065183 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:07:23.065372 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:07:23.065505 (+ 133us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:07:23.065953 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:07:23.066712 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:07:23.067948 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:07:23.068322 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:07:23.068437 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:07:23.068859 (+ 422us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:07:23.069289 (+ 430us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:07:23.069366 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 02:07:23.069746 (+ 380us) server_negotiation.cc:299] Negotiation successful
0809 02:07:23.069823 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":387,"thread_start_us":94,"threads_started":1}
I20260809 02:07:23.073685 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 26291
I20260809 02:07:23.083506 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 26372
I20260809 02:07:23.092096 1795 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRWTcX9/build/debug/bin/kudu with pid 26452
2026-08-09T02:07:23Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3387 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6668 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (301087 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST